Hi @sbrown,
Great question.
In principle, the Xcalar platform was architected to use open interfaces and open artifacts with a proprietary engine. This architecture ensures both openness and scale-out performance.
To unplug Xcalar design, here's what you'd need to do:
Data
-
No changes. Xcalar accesses your data in place. Worst case would be needing to export some resulting datasets from Xcalar Design, to one of your current data storage locations.
Algorithms
- UDFs – No changes. Code you write for Xcalar UDFs is 100% Python, with no proprietary additions.
- Dataflows – Some conversion. A dataflow is the algorithm created by your modeling operations to produce a table. Because there is no open standard for encoding a dataflow, we encode these as openly as possible: JSON-structured text containing standard operators in clean language (for example, filters, joins, splits). You presumably would task a developer with parsing and automating the conversion of a Xcalar Dataflow into another platform's syntax. We anticipate making this process even easier in the future, as the Xcalar platform's value is in the performance efficiencies of executing batch dataflows.
UI
- Coding Environment – No changes. As we integrated Jupyter Notebook as an integrated development environment to write UDFs, you can simply download the Notebooks to get not just code, but context decisions made when architecting your functions.
- Modeling – No changes. Xcalar's modeling uses a simple spreadsheet paradigm, so users aren't locked into using a proprietary UI with a steep learning curve.
@sbrown , did that answer your questions?
Mark
