O que a vaga pede
Data Engineer
Brazil , Remote
Type of Hire-Long Term Contract
Rate best we have $30 per hour USD Without any benefit
Role Overview
We are seeking a Senior Data & AI Infrastructure Engineer to lead the architecture, deployment, and automation of our modern data platform and agentic AI systems on Google Cloud. In this role, you will bridge data engineering, cloud infrastructure (IaC), and advanced Generative AI agents. You will design serverless data pipelines using BigQuery and Dataform while building and deploying AI Agents powered by Google Cloud’s Agent Development Kit (ADK) and Vertex AI Agent Engine.
Required Skill Set & Technical Requirements
1. Infrastructure as Code (IaC) & Cloud Management
Terraform:
Proficiency in writing modular, clean, and testable Terraform code (HCL) to provision Google Cloud Platform (GCP) resources.
Hands-on experience automating the lifecycle and deployment of BigQuery datasets, Dataform workspaces, and Vertex AI Agent Engine/Reasoning Engine resources via Terraform.
Deep understanding of CI/CD integration for Terraform state management, plan validation, and zero-downtime infrastructure updates.
2. Modern Data Platform & Engineering
BigQuery:
Advanced SQL mastery, partition/clustering strategies, materialization strategies, and performance optimization for enterprise data warehouses.
Experience with BigQuery ML, remote functions, and integration with generative AI toolsets.
Dataform:
Expertise in organizing, executing, and testing data transformation pipelines using SQLX and JavaScript within Dataform.
Ability to configure data assertions, execution dependencies, and incremental load strategies.
Experience using automated AI assistants or Data Engineering Agents to optimize data transformation workflows.
3. AI Agents & Software Engineering
AI Agents & ADK (Google Agent Development Kit):
Core understanding of Agentic workflows: reasoning loops, function calling/tooling, prompt orchestration, dynamic skill sets (SkillToolset), and context management.
Hands-on experience building conversational, analytics, or task automation agents using the Google Cloud ADK framework.
Proven ability to equip ADK agents with custom toolsets (e.g., querying BigQuery, triggering Dataform runs) and package them for production using Vertex AI Reasoning Engine / Agent Engine.
Python:
Senior-level Python proficiency (Python 3.10+), clean code architecture, Object-Oriented Programming (OOP), and async programming.
Experience with libraries standard to agent development: google-adk, google-cloud-aiplatform, cloudpickle, pydantic, and pandas.
Ability to write production-grade API wrappers, custom agent tool definitions, and automated testing suites.
Key Responsibilities
Data Pipeline Automation: Build end-to-end data transformation models in BigQuery using Dataform, maintaining enterprise-grade testing and data quality assertions.
Agent Architecture: Design and deploy production-ready AI agents using Python and Google ADK to auto-analyze data, interact with BigQuery, and automate enterprise workflows.
Automated Provisioning: Use Terraform as the single source of truth to manage all GCP resources—from data warehouses and IAM permissions to Vertex AI Agent instances.
Evaluation & Optimization: Establish evaluation frameworks (e.g., Vertex AI GenAI Evaluation) for agent performance and accuracy.
Collaboration: Partner with Data Science, Analytics, and DevOps teams to align data modeling with agentic capabilities.
Preferred Qualifications
GCP Professional Data Engineer or Google Cloud Professional Cloud Architect certification.
Experience deploying agent dependencies using cloudpickle artifact packaging for Vertex AI.
Familiarity with vector databases, embeddings, and hybrid retrieval-augmented generation (RAG) architectures.