Thanks! I got additional conflict from your professional services engineer. Because you have a complex, long running dataflow, I understand why you might want to halt it.
The two things you want, may be in conflict:
You want to yield errors into the dataset.
You want to halt the dataflow (and access the errors).
If you halt a batch dataflow upon import, you won't be able to access any results tables from the dataflow, as the batch will fail before they are generated. If this is your desired outcome, you will want to log the errors using the Python native logging facility, as I mentioned during an earlier post.
If you allow the import to succeed, then you won't be able to stop the dataflow from processing based on an import error. If this is your desired outcome, export a 2nd table from your dataflow, containing the erroneous rows from this import.
To halt the workflow, run the import as Low Error Tolerance (The Error Tolerance setting is in Advanced Settings. Low is the default for that setting) and raise the exception you throw.
Did that answer all of your questions regarding your import UDF? This is an interesting set of concerns, so I'm glad we get to discuss this and share with other Xcalar customers!