I was building a pipeline which was taking data from SFTP folder and sink to Cleansed layer where file format was .parquet. As soon as validated the pipeline got the below error, where the error message is: 'Sink dataset filepath can not contain a file name, please remove the file name from {DatasetName}.
You will find error like this:
Fig 1: Sink dataset error
If you find this error, may get confuse at first since you have file name at two different places. One is at Sink activity settings and other one is at the data set.
Let's guide you to how to find those settings and fix this error. If you look at the Settings for Sink activity, you will find like below fig: 2.
Fig 2: Settings of Sink activity
Fig 4: remove file name from Data set
Now the error will disappear. Lesson learned, look at the error message more than once and locate the correct place to fix the issue.
No comments:
Post a Comment