In principle, the Xcalar platform was architected to use open interfaces and open artifacts with a proprietary engine. This architecture ensures both openness and scale-out performance.
To unplug Xcalar design, here's what you'd need to do:
No changes. Xcalar accesses your data in place. Worst case would be needing to export some resulting datasets from Xcalar Design, to one of your current data storage locations.
- UDFs – No changes. Code you write for Xcalar UDFs is 100% Python, with no proprietary additions.
- Dataflows – Some conversion. A dataflow is the algorithm created by your modeling operations to produce a table. Because there is no open standard for encoding a dataflow, we encode these as openly as possible: JSON-structured text containing standard operators in clean language (for example, filters, joins, splits). You presumably would task a developer with parsing and automating the conversion of a Xcalar Dataflow into another platform's syntax. We anticipate making this process even easier in the future, as the Xcalar platform's value is in the performance efficiencies of executing batch dataflows.
- Coding Environment – No changes. As we integrated Jupyter Notebook as an integrated development environment to write UDFs, you can simply download the Notebooks to get not just code, but context decisions made when architecting your functions.
- Modeling – No changes. Xcalar's modeling uses a simple spreadsheet paradigm, so users aren't locked into using a proprietary UI with a steep learning curve.
@sbrown , did that answer your questions?