ADB Pyspark Engineer

Diverse Lynx

Posted: 4 months ago
Openings: 10
Applicants: 0

Job Description

Hiring for ADB Pyspark Engineer - Pune/Mumbai/Bangalore/Kolkata
Job Summary: We are seeking a skilled PySpark Data Engineer to join our data team. You will be responsible for designing, developing, and optimizing scalable data pipelines using Apache Spark and Python (PySpark).
Key Responsibilities:

Develop and maintain robust, scalable ETL/ELT data pipelines using PySpark and Apache Spark.
Design and implement data integration solutions from various structured and unstructured data sources (e.g., SQL/NoSQL databases, APIs, flat files, streaming).
Optimize and tune Spark jobs for performance and scalability.
Work with cloud platforms such as AWS, Azure, or GCP for data storage and processing (e.g., S3, EMR, Databricks).
Ensure data quality, integrity, and reliability by implementing validation, testing, and monitoring tools.
Collaborate with stakeholders to understand data requirements and translate them into technical solutions.
Document code, architecture, and data workflows.
Support data operations, troubleshooting, and performance monitoring.