Full employer posting
We are seeking a skilled Database Engineer to join our team within the Windreich Department of AI and Human Health and Institute for Critical Care Medicine (ICCM). In this role, you will be responsible for designing, implementing, and maintaining robust datamarts , data warehouses, databases and data visualization platforms for healthcare data. You will utilize your expertise in database management systems, to work with both current ETL/Data Warehousing and future Big Data/Streaming/Pipeline architectures.
The focus will be on choosing optimal solutions to use for these purposes, then implementing, maintaining, and monitoring them, always being mindful of the overarching goal of accelerating translational research and improving clinical care. Responsibilities Facilitate data collection from a variety of different sources, getting it in the right formats, assuring that it adheres to data quality standards, and assuring that downstream users can get that data quickly and with a common standard interface.
Ensure that data streams/pipelines are scalable, repeatable, and secure, and can serve multiple users. Design and develop a comprehensive datamart framework and data visualization platforms for healthcare data, ensuring scalability, efficiency, and data integrity. Manage and administer databases on relational database management systems such as Postgres or Microsoft SQL Server, optimizing performance and ensuring high availability.
Develop and implement Extract, Transform, Load (ETL) processes to seamlessly integrate healthcare data from sources including Epic Inc's Caboodle and Clarity databases into the data mart. Apply AI agents to data engineering work, including pipeline development, SQL generation, data quality checks, documentation, and natural language access to databases. Responsible for creating the infrastructure that provides insight from raw data and handles diverse sources of data seamlessly.
Additional responsibilities include developing prototypes and proof of concepts for the selected solutions, and implementing complex big data projects with a focus on collecting, parsing, and managing large sets of data using multiple platforms to allow for Operational, Research and Data Science initiatives. Translate business requirements into modern data pipeline solutions. Create centralized documents and diagrams of all solutions.
Design and implement monitoring, backup, and disaster recovery of data systems. Create a data catalog store of all metadata. Responsible for the integrity and security of data in all forms of storage throughout the Data Architecture.
Ensure compliance with the Institutional Review Board and HIPAA to follow all applicable policies and procedures. Assist in the development of standards and procedures affecting data management, design and maintenance. Documents all standards and procedures.
Possess an extremely flexible attitude. Willing to work with multiple types of technologies and languages with an open mind and without technology bias. Continuous interest in updating skill sets and knowledge of trends in the Big Data Technology space.
Work closely with cross-functional teams including data scientists, healthcare providers, and IT professionals to understand data requirements, develop solutions, and support data-driven decision-making. And other duties as assigned. Qualifications Bachelors degree in Computer Science or a related discipline; Advanced degree preferred. 4+ years relevant professional development experience, preferably in a LINUX environment.
Strong SQL and NoSQL Database Knowledge: Oracle, PostgreSQL/MYSQL, and Mongo DB (or similar). Proficiency with at least 2 programming languages among Scala/Python/ Java. Must be flexible and fast to pick up new languages.
Proficiency in Restful service development, preferably with Node JS, Django and PHP. Experience with micro-services and SOA. Strong SQL and NoSQL Database Knowledge: Oracle, PostgreSQL/MYSQL, and Mongo DB (or similar).
Proficiency on installation and configuration of big data software and technology Knowledge of Hadoop, Spark, Kafka and other big data technology stacks and streaming tools. Familiarity with and the ability to leverage a wide variety of open source technologies and tools. Working knowledge of cloud architecture and implementation onAzure or AWS, is a big plus.
Experience with serverless computing, creating VMs, cloud security, and other cloud services is also a big plus. Experience working in an Agile methodology. Experience working with JIRA is a plus.
Preferred: Bachelor's degree in Computer Science , Information Technology, or a related field. Master's degree preferred. Strong proficiency in ETL processes, data warehousing, and data integration techniques with proven experience in database development and management Strong SQL skills and experience with relational databases such as Microsoft SQL Server, Oracle, or PostgreSQL.
At least 2 years of hands-on experience building and maintaining ETL/ELT pipelines from Epic Clarity and Caboodle, with working knowledge of their data models. Epic Clarity or Caboodle Data Model certification preferred. Proficiency in Python and experience with modern data engineering tools, including cloud lakehouse platforms (Databricks, Snowflake, or Microsoft Fabric), orchestration and transformation tools (Airflow, dbt ), and Git-based CI/CD.
Experience using AI agents or LLM-based tools for database and data engineering tasks is strongly preferred. Working knowledge of cloud architecture and implementation on Azure or AWS, is a big plus. Experience with serverless computing, creating VMs, cloud security, and other cloud services is also a big plus.
Excellent analytical and problem-solving skills with a keen attention to detail. Effective communication skills and ability to collaborate in a team environment.
Full time · Day shift
About the role
Facilitate data collection from a variety of different sources, getting it in the right formats, assuring that it adheres to data quality standards, and assuring that downstream users can get that data quickly and with a common standard interface.
