Release Date: June 1, 2018
In this release, we have made significant improvements to how you manage and work with Xcalar workbooks.
We re-architected the Xcalar workbook, redesigned the Xcalar Design Workbook page to enable you activate multiple workbooks for modeling and analyzing data, and added downloading and uploading functions for sharing workbooks with others or for using with another instance of Xcalar.
Xcalar now supports SQL queries on structured and semi-structured data. This is achieved by creating a SQL Table Alias from the new SQL extension and using this alias as your table name in your SQL query statements.
The new Xcalar Design SQL panel contains two sections: the Xcalar Design SQL code editor, which is for entering and applying SQL to your data, and the Tables and Schemas available to SQL section, which is for managing your SQL Table Aliases, as shown in the screenshot below:
Change Data Capture and Rollback with IMD
Xcalar Design’s new Published Table feature supports the tracking of continuous data changes, which enables you to revert to any data state for any given date and time from the new IMD screen and view the results in an automatically created Xcalar table.
This is achieved by creating a Published Table and applying your data updates to the Published Table. Then, from the new IMD screen in Xcalar Design, you can instantly see when changes were applied to the Published Table and analyze and rollback changes on all your data to a specific point-in-time.
More Features and Enhancements in Xcalar 1.4.0
The following sections describe the new features and enhancements for Xcalar release 1.4.0:
This section describes the enhancements and new features for Xcalar workbooks.
Switching from one active workbook to another at any time has never been easier. With just one mouse click anywhere on the workbook’s card and accepting the confirmation message, the workbook is displayed in the current workbook browser tab, while still keeping the other workbooks that you are working on active. Or you can open the workbook in a new browser tab by selecting the ellipsis icon.
A downloaded workbook is a self-contained repository of all the code and operations associated with your import and modeling work. It is uniquely associated with the user-defined functions (UDFs) and Jupyter Notebooks that were used during creation. When downloading a workbook, all associated content, such as UDFs and Jupyter Notebooks are contained in the workbook’s archive file format.
This section describes the security enhancements and new features for user-defined functions (UDFs).
Increased UDF protection
UDFs are now local to a workbook or batch dataflow and can only be opened as read-only when accessed using a different workbook. Previously, UDFs were global to a cluster.
NOTE: The only exception is the default UDF, which is not editable. Each cluster has a default UDF, which is a library of available functions that you can use and is stored in a workbook. It is not copied to a new cluster upon uploading a workbook or batch dataflow.
New UDF management organization enhancement
The UDF Manager tab now organizes UDFs by user and workbook or batch dataflow under the following headings:
- Current Workbook
- My Workbooks
- Other Workbooks
- Batch Dataflows
Jupyter Notebook enhancements
This section describes the security enhancements for Jupyter Notebook.
Jupyter Notebooks can only be opened as read-only by other Xcalar users.
Jupyter Notebooks are now created in the Xcalar workbook directory associated with the Jupyter Notebook. This makes finding Xcalar workbooks easier as your Jupyter Notebooks are tied to the Xcalar workbook.
Uploading a Jupyter Notebook automatically replaces the connection code snippet with one appropriate to your workbook.
Batch Dataflow enhancements
This section describes the enhancements for batch dataflows.
New advanced mode customization
Previously, only certain fields from the import, filter, and export operations were editable through the use of customization and parameterization. With this release, you can directly customize any operation in your batch dataflow within Xcalar Design.
NOTE: Parameterization is still limited to import, filter, and export operations in Xcalar Design.
Parameters are now global across all batch dataflows in a Xcalar cluster. You can now set the value of a parameter and all instances of the parameter across all batch dataflows. Your new value replaces the old value when the batch dataflow is run.
Additional Operations Support
Xcalar now supports the Except and Intersect operators, which behave similar to the equivalent operators in SQL:
- Using Intersect on two or more tables, returns the rows that are common between the tables.
- Using Except on two or more tables, returns the rows in the left most table that are not in the other tables.
Column Limit Increase
The limit for customized columns in a table has been increased to approximately 1023.
New Xcalar License Key Format
The Xcalar license key format has changed. Before upgrading, please contact your sales representative for a new Xcalar license key in the new format, as previous Xcalar licenses keys will not work.
Fixes in this Release
This section lists the issues reported and provides a brief description of the resolution.
Import UDFs were not applied, as configured, on datasets that consisted of multiple data source files, in a custom format, whose schema consisted of over 128 data fields. Because of the custom format, a UDF was used to change the format. This issue is now resolved.
Previously, the standard deviation calculation included field not found (FNF) values. With this release, only values that exist are included in the standard deviation computation.
For more information on all how to use the new features and enhancements in Xcalar Design, see the Xcalar Online Help documentation.