Here's a quick update to this information. Since this post, Xcalar added Jupyter Notebook to Xcalar Design. This reduces a developer's need to access server log files.
Instead of plumbing around the logs for UDF errors, a user can now write and test his/her UDFs in Jupyter Notebook and test using Jupyter's interactive shell. This gives users an instance of the same version of Python that the XD cluster is using, as well as the ability to apply a UDF to data from a table in your workbook.
If your map UDF wasn't developed in a Jupyter Notebook, you would test it as follows:
1. Create a new Jupyter Notebook. Upon creation, XD will add a Connect to Xcalar Workbook code snippet to the first cell of the Jupyter Notebook.
2. Publish a partial table to Jupyter Notebook using the table's context menu. This will add a Publish Table code snippet to the 2nd cell in your Jupyter Notebook.
3. Go to the built-in UDF editor and copy the Python module code for the UDF into the third cell.
4. At the end of the third cell, I'd add a Python loop test call for the function, which uses rows of data from the table, including any theoretical error condition.
5. Finally, I'd run the cells in order. (Connect, then Data, then UDF code.)
Using those steps, I would see errors and iterate on my UDF code within Jupyter, and then copy the Python module code back to the UDF editor, once I complete my changes. This will reduce the potential errors to be debugged using log files. Glad to go into more detail, if folks are having trouble with this.