BigData

Essential functions

• The responsibilities of this role revolve around efficient processing and management of large-scale datasets using Apache Spark with Scala.
• The candidate will be tasked with designing, developing, and maintaining data pipelines and workflows to support various data processing needs.
• Optimizing data ingestion, transformation, and storage processes to ensure reliable and scalable data processing is a key aspect of this role.
• Collaboration with cross-functional teams to understand business requirements, implement data solutions, and ensure data quality and integrity is also expected.
• Additionally, the candidate will contribute to performance tuning, troubleshooting, and monitoring of Spark applications to maximize efficiency and minimize processing time.

Qualifications

• 5+ years of experience
• Apache Spark, Scala, Distribute Systems, Parquet
• Proficiency in Apache Spark: Strong expertise in Apache Spark is essential, including hands-on experience with Spark Core and Spark SQL
• Apache Iceberg experience
• Programming Skills: Solid programming skills in Scala are crucial
• Big Data Technologies: Familiarity with big data technologies and frameworks such as Hadoop, Hive, and HDFS is important.
• Data Pipeline Development: Experience in designing, building, and maintaining data pipelines is required. This includes expertise in data ingestion, transformation, and integration, ensuring data quality and consistency throughout the pipeline.
• Distributed Systems and Cloud Platforms: Proficiency in working with distributed systems and cloud platforms like AWS, Azure, or Google Cloud is necessary
• Monitoring and Performance Tuning: Proficiency in monitoring and optimizing Spark applications for performance is important.

Would be a plus

4 or more years experience in Data Engineering, experience with Delta Lake + Snowflake

We offer

• Opportunity to work on bleeding-edge projects
• Work with a highly motivated and dedicated team
• Competitive salary
• Flexible schedule
• Benefits package - medical insurance, sports
• Corporate social events
• Professional development opportunities
• Well-equipped office

About us

Grid Dynamics (NASDAQ: GDYN) is a leading provider of technology consulting, platform and product engineering, AI, and advanced analytics services. Fusing technical vision with business acumen, we solve the most pressing technical challenges and enable positive business outcomes for enterprise companies undergoing business transformation. A key differentiator for Grid Dynamics is our 8 years of experience and leadership in enterprise AI, supported by profound expertise and ongoing investment in data, analytics, cloud & DevOps, application modernization and customer experience. Founded in 2006, Grid Dynamics is headquartered in Silicon Valley with offices across the Americas, Europe, and India.