Senior Data Engineer | AWS & GCP

🔵 Reach IT - Nearshore Solution! Our nearshore expertise sets us apart. We specialize in connecting experienced technology professionals with complex international engineering environments, supporting organizations in building scalable, reliable, and high-performance data platforms. We are looking for a Senior Data Engineer | AWS & GCP 🛡️ To join a Life Sciences Data Engineering project, supporting a pharmaceutical organization specialized in research, development, and manufacturing of innovative therapies. The project focuses on the evolution of an enterprise cloud data platform, integrating research, laboratory, manufacturing, and operational datasets to support analytics, data quality, and scientific decision-making. Skills 🎯 AWS: S3, Glue, EMR and Step Functions for ingestion, processing and orchestration; 🎯 GCP: BigQuery, Cloud Storage, Dataflow and Cloud Composer for analytical/DWH; 🎯 Python, SQL and PySpark for data transformations and distributed processing; 🎯 ETL pipelines for enterprise data integration and processing; 🎯 Data Lake & Data Warehouse architectures for scalable storage and analytical consumption; 🎯 Apache Airflow for pipeline orchestration and dependency management; 🎯 Data Quality & Governance for validation, lineage and traceability; 🎯 Terraform, Git & GitHub Actions for infrastructure automation and controlled deployments; 🎯 Performance Optimization for distributed workloads and analytical queries. Responsibilities ✅ Develop AWS-based ingestion and processing pipelines, using S3 to organize source datasets, Glue for extraction and transformation, and EMR for large-scale PySpark processing; ✅ Implement GCP analytical data pipelines, using Cloud Storage for staging, Dataflow for transformation workflows, and BigQuery for curated analytical datasets and reporting consumption; ✅ Build Python, SQL, and PySpark transformations to standardize, cleanse, validate, and combine data from research, laboratory, manufacturing, and enterprise systems; ✅ Design and maintain ETL workflows connecting data processing and analytical layers, managing schema evolution, incremental loads, data dependencies, and processing efficiency; ✅ Configure AWS Step Functions and Cloud Composer / Apache Airflow to coordinate pipeline execution, scheduling, monitoring, retries, and recovery mechanisms; ✅ Optimize Spark processing jobs and BigQuery workloads, improving partitioning strategies, query execution, storage efficiency, and cloud processing costs; ✅ Implement data quality checks and validation rules using AWS Glue Data Quality, PySpark and SQL, leveraging Google Cloud Dataplex for data quality monitoring, metadata management and lineage; ✅ Automate cloud infrastructure and data pipeline deployments through Terraform and GitHubActions; ✅ Collaborate with Data Architects, Analytics Engineers, and pharmaceutical domain teams to deliver reusable data models, reliable transformation frameworks, and scalable data products. #SeniorDataEngineer #DataEngineer #AWS #GCP #GoogleCloud #AmazonWebServices #Python #PySpark #SQL #BigQuery #AWSGlue #AmazonEMR #Dataflow #CloudComposer #ApacheAirflow #DataEngineering #ETL #ELT #DataPipelines #Pharmaceutical #LifeSciences #Remote #CloudDataEngineering #ReachIT