Create an Azure Storage Data Stream

Create a data stream and import objects from Azure Storage. There can be some latency between creating your data stream and object data availability in Data Cloud while data is transferred to the staging environment. The data stream detects the schema from your source file and displays the appropriate Data Cloud data type.

Complete these prerequisites:

• Make sure the Azure Blob Storage Connection is set up. For more information, see See Set Up an Azure Connection.

1. In Data Cloud, go to the Data Streams tab and click New.

   You can also use App Launcher to find and select Data Streams.

2. Select Microsoft Azure Blob Storage.

3. Select the applicable connector from the Connection dropdown.

4. Complete file and source details.

   Field Label	Description
   File Type	Select CSV or Parquet. For more information, see Supported File Formats and Delimiters.
   Import From Directory	The remaining folder path under the container that points to a file’s specific location. Place your source files in the directory because the data stream can’t recognize files stored in nested subdirectories.
   File Name	Name of the file that must be retrieved from the specified folder. The field is pre-populated with '*'. If no file is specified, the system chooses the first file found. After you create a data stream, it retrieves all files found in the directory. Wildcards are also supported, for example you can use *abc*.csv to indicate retrieval of all files containing “abc” in their name. Each time the stream runs, all files satisfying the wildcard are imported.
   Source	A label designating the external system from where data is sourced. Multiple data streams can use the same label for the Source.

5. Complete Object Details. You can create a data lake object (DLO) or choose to use an existing DLO.

   If you choose to create a DLO then refer to Naming standards for data lake objects. If you choose to use an existing DLO then refer to Using existing data lake object to create a data stream and familiarize yourself with the guardrails to be considered when using an existing DLO.

6. Select a data stream category and primary key. Add new formula fields if needed.

7. From the Data Space dropdown, select the applicable data space or the default data space.

8. Fill in deployment details and click Deploy.

Your data stream is created. Map your data stream to data model objects and start using your data for your Azure Storage use cases.