Create an SFTP Data Stream
Create a data stream in Data Cloud to start the flow of data from your SFTP-enabled server source.
- Make sure an SFTP connection is set up. See Set up an SFTP Connection.
-
In Data Cloud, on the Data Streams tab, click New.
You can also use App Launcher to find and select Data Streams.
-
Select Secure File Transfer (SFTP), and click Next.
-
Choose from the available SFTP connections.
-
Enter the import details.
Field Description Import from Directory Remaining folder path under the parent directory that points to a file’s specific location. File Name The file name to retrieve. The file name supports wildcards, and CSV file formats. For example, you can use abc.csv to retrieve all file names containing “abc.” Each time the stream runs, all files that match the name, including wildcards, are imported. Source The external system where data is sourced. Multiple data streams can have the same source. -
Click Next.
-
Create a data lake object (DLO) or use an existing DLO.
If you’re creating a DLO, see Naming standards for data lake objects. If you’re using an existing DLO, see Using existing data lake object to create a data stream.
-
If you’re creating a DLO, edit the fields identified in the table.
-
Enter a DLO label, API name, and object details.
Field Description Category Identify the type of data in the data stream. Primary Key Select a unique field to identify a record. Record Modified Field If data is received out of order, the system uses the record modified field as a reference point to determine whether to update the record. The record with the most up-to-date timestamp is loaded. Organization Unit Identifier Select the business unit to use for a record’s data lineage. -
(Optional) Add formula fields.
-
Click Next.
-
Enter the deployment details, and click Deploy.
Your data stream is created, and you can now map your DLO to the semantic data model to use the data in segments, calculated insights, and other use cases.