O que a vaga pede
We're Hiring: Databricks + Azure Data Engineer
Location: LATAM based (Remote)
Are you a data engineering pro who lives and breathes Azure Databricks and the Lakehouse? We're looking for an experienced Databricks + Azure Developer to design, build, and optimize scalable data engineering solutions on Azure. If you love building high-performance ETL/ELT pipelines and implementing Medallion architecture, let's talk!
What You'll Do
• Design, develop, and maintain data pipelines using Azure Databricks and PySpark
• Build and orchestrate ETL/ELT workflows with Azure Data Factory (ADF)
• Develop ingestion & transformation pipelines using ADLS Gen2, Delta Lake, and Spark SQL
• Implement Medallion architecture (Bronze/Silver/Gold), incremental loads, CDC, MERGE, and SCD Type 1/2
• Optimize Spark jobs via partitioning, caching, file-size optimization, and performance tuning
• Configure and maintain Databricks Workflows/Jobs, clusters, and workspaces
• Build data quality checks, validation, error handling, monitoring, and reconciliation
• Leverage Unity Catalog for governance, lineage, RBAC, and secure data access
• Integrate Databricks with Azure services (ADLS, Azure SQL, APIs, JDBC)
• Implement CI/CD with Azure DevOps & Git; automate IaC with Terraform or ARM/Bicep
• Troubleshoot production pipelines, run root-cause analysis, and ensure SLA compliance
• Collaborate with architects, analysts, and BI teams in an Agile environment
What You Bring
• 7–10+ years in Data Engineering or Azure cloud data development
• Strong hands-on with Azure Databricks, PySpark, Spark SQL, and Delta Lake
• Proven experience building scalable ETL/ELT pipelines with ADF
• Solid grasp of ADLS Gen2, Lakehouse architecture, and distributed data processing
• CI/CD (Azure DevOps), Git, YAML pipelines, and IaC (Terraform/ARM/Bicep)
• Strong SQL skills with performance-tuning expertise
• Experience in Agile/Scrum environments
Nice to Have
• Databricks platform administration (clusters, jobs, workspace config)
• Unity Catalog, data governance, lineage & RBAC know-how
• Databricks integration with APIs, JDBC sources & external systems
• Familiarity with Datadog/Grafana and cost optimization