Senior Software Engineer- Data

Job Description

Key Responsibilities

• Build
  scalable batch and streaming pipelines using Python, PySpark, and
  SQL.



• Develop
  and maintain ETL/ELT workflows and data ingestion frameworks.



• Design
  dimensional data models (fact/dimension, star schema, 3NF).



• Write
  and optimize complex SQL transformations for large datasets.



• Ensure
  data quality, validation, lineage, and governance best practices.



• Tune
  Spark jobs and SQL queries for performance and cost.



• Work
  with AWS services (S3, Glue, EMR, Lambda, Step Functions).



• Implement
  CI/CD, testing, and production deployment of data workflows.



• Monitor,
  log, and troubleshoot data pipelines in production.



• Participate
  in architecture discussions and drive technical decisions.



• Collaborate
  with analytics, product, and data science teams.



• Mentor
  junior engineers and promote best practices.

Required Skills Qualifications

• 5+
  years of experience in data engineering or related roles.



• Strong
  Python development experience.



• Advanced
  SQL (analytical queries, performance tuning).



• Expertise
  in PySpark (DataFrame API, RDD concepts, optimization).



• Solid
  understanding of data modeling for OLTP/OLAP.



• Proven
  ETL/ELT pipeline design experience.



• Cloud
  experience (AWS preferred; similar GCP/Azure OK).



• Knowledge
  of data warehousing concepts and distributed systems.



• Git
  and CI/CD experience.



• Strong
  debugging, optimization, and problem-solving skills.



• Ability
  to work with large-scale datasets.

Preferred Skills

• Airflow/Dagster/Prefect



• Kafka/Kinesis
  or streaming systems



• Docker/Kubernetes



• IaC:
  CDK, Terraform, CloudFormation



• Delta
  Lake, Iceberg, or Hudi



• ML
  feature pipelines



• Monitoring
  tools: CloudWatch, Datadog, Prometheus, ELK



• System
  design and architectural experience

Soft Skills

• Clear
  communication and documentation



• Ownership
  mindset



• Mentorship
  capability



• Collaboration
  with cross-functional teams



• Detail
  orientation and structured thinking

Why Work With Us

• High-impact
  ownership of modern data platforms



• Exposure
  to advanced cloud, Spark, and lakehouse technologies



• Collaborative
  culture focused on learning, innovation, and engineering excellence



• Opportunity
  to shape data architecture and strategy

More Info

Education