Release Date: March 21, 2018
File Browser Redesign
Xcalar redesigned the File Browser page to help you locate, manage, and work with your data sources faster and more efficiently. This was achieved by introducing more options to help you locate your data, new icons to distinguish data formats, extra data source descriptions that help you choose the correct data, a new feature for displaying your data in hex, and added important new features that give you the ability to select multiple folders and files and import as either one or multiple datasets, with one single click.
More Control Over Who Accesses your Datasets
Xcalar takes security very seriously. In this release, we changed who can access your datasets
to give you more control over your analysis environment. Now you can choose whether your datasets are private and can only be used by you, or whether your datasets can be shared with other Xcalar Design users on the same cluster.
Create Datasets from CSV formats with Data Type Smart Casting
When creating a dataset from CSV formatted data, our new CSV data type smart cast feature detects the CSV data types for you and automatically assigns the correct data types to the data fields in the newly created dataset. To help save you further time, so that you can quickly analyse your data, you can manually re-cast the data type and change the name of a data column from the Dataset Preview window, before you create a table.
More Features and Enhancements in Xcalar 1.3.1
The following sections describe the new features and enhancements for Xcalar release 1.3.1:
Browse Data Source Page Enhancements
This section describes the enhancements and new features in the Browse Data Source page.
Improved Data Source Selection
With this release, users can now select multiple files and/or folders from the data target. They are displayed in a list format and the following can be enabled:
- Data is imported from a folder recursively by selecting the checkbox by the folder path in the Selected Files / Folders list.
- Data is imported as either one dataset or multiple datasets, by clicking one of the slider bar options.
File Browser Search Options
Xcalar Design’s file browser now supports the following regular expressions and matching options that enable users to find datasets and other objects faster:
Clearer and more Distinguishable File Format Icons
With this release, the file format icons have been redesigned to give users a clearer visual understanding of the file format without having to know the file extension.
Extra Data Source Details
Users can now see more information about their data source, such as the data source format, last modified date, and size. Users also have the option to preview the data source’s raw data, and can switch from the file browser mode to the hex dump mode of a selected data source file.
Import Data Source Page Enhancements
This section describes the enhancements and new features in the Import Data Source page.
CSV format enhancements
Before creating a dataset from CSV formatted files, users can:
- Rename fields.
- Change a field’s data type.
Jupyter Notebook integration enhancements
- Users can apply a user-defined function (UDF) directly into an operation and on a table.
- Users have the option of parsing the dataset with a UDF from the Format section. The Format list contains the Custom Format type, which enables users to then select the UDF module.
- Users can write and test a parser directly in Jupyter Notebook. With the ability to now parse a UDF from the Import Data Source page, users can also write and test a parser in Jupyter Notebook before they import the dataset by clicking the WRITE PARSER USING JUPYTER button.
- Users have the option to debug their existing UDFs while previewing the UDF’s operation on the data source they would like to import.
New Advanced Options section
This new section contains options for managing data source imports.:
- To help identify data and work with datasets created from your data source, this section offers options that create columns in their datasets that help the user work more efficiently with their data.
- To make error handling more transparent with more robust parsing, this section offers options for how users would like Xcalar to handle errors while importing their files.
New File Format Types and Changes to existing File Types
- Xcalar now supports native XML data and Parquet formatted datasets and files.
- When importing Excel files, Xcalar now requires users to enter the worksheet number from the excel file workbook that they want to import.
For more information on how to work with the new and updated format types in Xcalar Design, see the Xcalar Online Help documentation.
- Users can view the progress of the import, when a UDF (inbuilt or external) is applied to the dataset. The progress icon also contains an option to cancel the import.
- Users can choose which file to preview amongst their selections, from the Previewing list.
Data Targets and Dataset Enhancements
This section describes the enhancements and new features in the Data Target panel, Dataset panel, and Dataset Preview page.
With this release users can now create private or shared datasets. By default, all the datasets created by a user are private. Users can make their datasets available to other users by selecting Share in the dataset options. This option creates a shared dataset and adds it into a dedicated shared folder for other users to use who are on the same cluster.
NOTE: Dataset sharing is either turned on or off by an admin. If it’s on, all users can share and unshare their datasets for others to use.
So what happens to datasets that have already been shared when the admin turns sharing off? Datasets that are already shared will remain shared until their respective users unshare them. But no new datasets can be shared.
Data Target Creation
Previous Xcalar releases allowed the creation of data targets by any Xcalar user. Starting from release 1.3.1, only administrators will be able to configure and create data targets. A data target contains information about the data source location, access to the location, and the data source format, which can be located on one or multiple storage platforms and contain secure access privileges.
New Advanced Section and Options
- Users can choose to have Xcalar alert them when an error occurs during an import. In the Dataset Preview window, an error icon is displayed next to the number of records in the dataset.
- Clicking the Error icon, opens the Dataset Import Error window, which displays the error file’s filename and the error message that describes the error. Users also have the option to download the error log.
- Clicking the CLICK TO VIEW button, opens the Files / Folders window, which displays a directory tree of all the files and folders within the dataset. To view files or folders of interest, expand or collapse folders.
- Users can cancel the creation of a dataset by clicking the CANCEL button under the progress icon.
With this release, users can now sort multiple columns by clicking the sort option above the header. Where, the most recent sort is given the highest priority and is sorted first, followed by the next most recent sort, and so on. The sort priority level is shown by the number near the sort icon.
In previous Xcalar releases, the field-not-found (FNF) values were removed during a sort operation. With this release, when a user sorts a column any FNFs values that maybe present in the column are retained. Where, sorting the column by ascending order will bring the FNFs to the top and a descending sort would do the opposite. This gives users the option to filter FNFs when required.
In previous Xcalar releases, when a user triggered an operation and then needed to remove specific columns from the DATA column they had to exit the operation’s panel.
With this release, users can pull columns from a data cell during an operation without having to exit the operation’s panel.
For more information on all how to use the new features and enhancements in Xcalar Design, see the Xcalar Online Help documentation.