Data Engineer Prophecy (RARR Job 5030)

1. Prophecy provides an easy-to-use visual interface to design and run data pipelines at scale

2. Prophecy is a complete product that provides visual development for Spark, Databricks, visual scheduling on Airflow, and metadata management with search and column-level lineage

3. Prophecy generates code on Git, tests, and enables CI/CD, bringing software engineering best practices to building data pipelines

4. Prophecy is extensible – users can add new visual components by writing Spark code templates

5. Prophecy works with large-scale and very complex workloads

6. Prophecy enables a wide set of data users to build data pipelines on Spark

7. Prophecy replaces ETL products as organizations migrate from legacy on-premises to the cloud

8. Prophecy runs within your network

Skills

• Mandatory Skills: MDM Conceptual, Profisee (MDM)