Backend Engineer - Search
Atomicwork Inc7 months ago
Bengaluru, IndiaMid Level
Responsibilities
- Design, develop, and maintain data pipelines for enterprise search applications.
- Implement data ingestion from databases, file systems, and APIs.
- Develop data transformation and enrichment processes for indexing.
- Integrate with search platforms to index and update data efficiently.
- Ensure data quality, consistency, and integrity throughout the pipeline.
- Monitor pipeline performance and troubleshoot issues.
- Collaborate with data scientists, engineers, and product managers.
- Implement security measures for sensitive data during processing and storage.
- Document pipeline architecture, processes, and best practices.
- Stay current with data engineering and enterprise search developments.
Requirements
- Proficiency in Python, Java, or Scala.
- Strong experience with data pipeline frameworks and tools such as Apache Airflow and Apache NiFi.
- Experience with Elasticsearch or OpenSearch.
- Familiarity with data ingestion, transformation, and indexing processes.
- Understanding of enterprise search concepts, including crawling, indexing, and query processing.
- Knowledge of data security and access control best practices.
- Experience with AWS, GCP, or Azure and related backend or integrations services.
- Familiarity with Model Context Protocol is a strong plus.
- Strong problem-solving and analytical skills.
- Excellent communication and collaboration skills.
Benefits
- The position is based in the Bengaluru office.
- Unlimited sick leave and 15 days off annually.
- Health insurance with up to 75% of the premium covered.
- Flexible allowances with reimbursement options.
- Annual company outings.
- Convenient offices across five cities.