Data Engineer - Databricks & PySpark

Location: Sydney
Duration: 6 months
Start: ASAP

I am working with an organization that's ready to take a product from proof-of-concept (PoC) to a fully operational, production-grade managed service. This is a hands-on role for a Data Engineer who thrives in fast-paced environments and knows how to turn prototypes into scalable, reliable solutions.

What You'll Be Doing

• Take ownership of an existing PoC and lead its evolution into a production-ready service.
• Build, refine, and scale data pipelines and outputs to support ongoing monitoring and management.
• Collaborate closely with developers and stakeholders to ensure smooth delivery within a tight 6-month timeline.
• Apply best practices in CI/CD and production deployment to ensure stability and maintainability.

What We're Looking For

Must-Have Skills

• Python: Strong to advanced proficiency, especially working with PySpark.
• Databricks: Solid experience with Unity Catalog and Delta Live Tables (intermediate to advanced).
• RESTful APIs: Advanced experience managing client credential requests using Python.
• CI/CD: Deep understanding of best practices; confident in setting up and maintaining pipelines.
• End-to-End Development: Experience pushing solutions into production environments.

Nice-to-Have Skills

• SQL: Useful for data manipulation and validation.
• Microsoft Graph API: Great if you've worked with it, but general RESTful API experience is fine

if this is an opportunity that you are interested in please reach out and we can hop on a call!