Azure Data Engineer / Databricks Data Engineer Job at ACS Consultancy Services, Austin, TX

RUxDZUNBYUNKR3dTeUdQMnhvMjZlbnRnQ3c9PQ==
  • ACS Consultancy Services
  • Austin, TX

Job Description

Job Title: Azure Data Engineer / Databricks Data Engineer

Location: Austin, TX (Remote)

We are currently seeking candidates who meet the following qualification

Mandatory Qualifications:

  • Implement ETL/ELT workflows for both structured and unstructured data.
  • Automate deployments using CI/CD tools.
  • Collaborate with cross-functional teams including data scientists, analysts, and stakeholders.
  • Design and maintain data models, schemas, and database structures to support analytical and operational use cases.
  • Evaluate and implement appropriate data storage solutions, including data lakes (Azure Data Lake Storage) and data warehouses.
  • Implement data validation and quality checks to ensure accuracy and consistency.
  • Contribute to data governance initiatives, including metadata management, data lineage, and data cataloging.
  • Implement data security measures, including encryption, access controls, and auditing; ensure compliance with regulations and best practices.
  • Proficiency in Python and R programming languages.
  • Strong SQL querying and data manipulation skills.
  • Experience with Azure cloud platform.
  • Experience with DevOps, CI/CD pipelines, and version control systems.
  • Working in agile, multicultural environments.
  • Strong troubleshooting and debugging capabilities.
  • Design and develop scalable data pipelines using Apache Spark on Databricks.
  • Optimize Spark jobs for performance and cost-efficiency.
  • Integrate Databricks solutions with cloud services (Azure Data Factory).
  • Ensure data quality, governance, and security using Unity Catalog or Delta Lake.
  • Deep understanding of Apache Spark architecture, RDDs, DataFrames, and Spark SQL.
  • Hands-on experience with Databricks notebooks, clusters, jobs, and Delta Lake.

Preferred Qualifications:

  • Knowledge of ML libraries (MLflow, Scikit-learn, TensorFlow).
  • Databricks Certified Associate Developer for Apache Spark.
  • Azure Data Engineer Associate certification.

Responsibilities:

  • Build and maintain robust ETL/ELT pipelines for structured and unstructured data.
  • Develop, deploy, and monitor data solutions in Azure and Databricks environments.
  • Collaborate with cross-functional teams to support analytics, reporting, and data science initiatives.
  • Design and optimize data models, schemas, and storage solutions for performance and scalability.
  • Implement data validation, quality checks, and security measures to ensure reliable and compliant data.
  • Maintain metadata, data lineage, and cataloging as part of data governance initiatives.
  • Troubleshoot, debug, and optimize Spark jobs for efficiency and cost-effectiveness.
  • Contribute to continuous improvement through automation of deployments and adoption of DevOps practices.


    If you meet these qualifications, please submit your application via link provided in LinkedIn.
    Kindly do not call the general line to submit your application.

Job Tags

Remote work,

Similar Jobs

Barnhart

Heavy Haul Truck Driver Job at Barnhart

 ...Pulling Super Loads, Local and Over-the-road hauls Pulling over dimensional loads...  ...Step Deck, Double and Triple Drop Loads Driver will work on a team transporting super loads...  ...maintenance. Ability to run heavy equipment is a plus Flexibility to travel... 

FedEx Services

Courier/swing drvr/dot-4 Job at FedEx Services

 ...functions. Performs other duties as assigned. Minimum Education High school diploma/GED. Minimum Experience Six (6) months courier experience preferred. Must be able to demonstrate area knowledge. Knowledge, Skills, and Abilities Ability to lift 50 lbs.... 

Community College of Philadelphia

Student Success Navigator, Catto Scholarship Job at Community College of Philadelphia

 ...with the entire Catto Scholarship team, Enrollment Management, Educational Support Services, and other areas at the College to...  ...associate-degree-granting institution which provides access to higher education for all who may benefit. Its programs of study in the... 

Atrium Hospitality

Breakfast Cook Job at Atrium Hospitality

**Hotel:**Rogers Embassy Suites3303 Pinnacle Hills PkwyRogers, AR 72758Breakfast CookFull time**Atrium SPIRIT - where teamwork, passion and appreciation ignite service excellence****What's in it for you?**The **Atrium SPIRIT** is a belief in the power of **_Service... 

MileHigh Adjusters Houston Inc

Independent Insurance Claims Adjuster in El Paso, Texas Job at MileHigh Adjusters Houston Inc

 ...IS IT TIME FOR A CAREER CHANGE? Independent Insurance Claims Adjusters Needed Now! This is a HUGE opportunity for you, since there is currently a HIGH DEMAND for NEW ADJUSTERS AND NEW STORM ADJUSTERS! Are you actively working as a Licensed Claims...