Data Pipeline Script Creator
Create a scalable data pipeline with this AI prompt, focusing on memory efficiency and real-world data challenges.
What this prompt does
- Develops a comprehensive, production-ready data loading script in Python.
- Focuses on memory efficiency and scalability for handling large datasets.
- Implements robust error handling and logging for real-world data challenges.
How to use this prompt
- Fill in the [INSERT YOUR DATA FORMATS], [INSERT DATA SIZE], [INSERT DATA LOCATION], [INSERT AVAILABLE RAM], and [INSERT ANY KNOWN ISSUES] placeholders with your specific data details.
- Example: "My data formats are CSV and JSON. My approximate data size is 5GB. My data location is on S3. My available memory is 16GB. My specific data challenges include encoding problems and missing values."
Free prompt — copy, paste and run. Part of the Coding prompts collection in the God of Prompt library.