Data Engineer - RT Big Data Systems

<b>Description</b><br><p><strong><em><u>abra R&amp;D is looking for a Data engineer!</u></em></strong></p><p>We are looking for a <strong>Data Engineer for RT Big Data Systems</strong> to join the team and design and deploy scalable, standardized, and maintainable data pipelines that enable efficient logging, error handling, and real-time data enrichment. The role requires strong ownership of both implementation and performance</p><p>The role includes:</p><ul><li>Optimize Splunk queries and search performance using best practices</li><li>Build and manage data ingestion pipelines from sources like Kafka, APIs, and log streams</li><li>Standardize error structures (error codes, severity levels, categories)</li><li>Create mappings between identifiers such as session ID, user ID, and service/module components</li><li>Implement real-time data enrichment processes using APIs, databases, or lookups</li><li>Set up alerting configurations with thresholds, modules, and logic-based routing</li><li>Collaborate with developers, DevOps, and monitoring teams to unify logging conventions</li><li>Document flows and ensure traceability across environments</li></ul><br> <b>Requirements</b><br><ul><li>Minimum 3 years of hands-on experience in Splunk – Mandatory</li><li>Proficient in SPL, data parsing, dashboards, macros, and performance tuning – Mandatory</li><li>Experience working with event-driven systems (e.g., Kafka, REST APIs) – Mandatory</li><li>Deep understanding of structured/semi-structured data (JSON, XML, logs) – Mandatory</li><li>Strong scripting ability with Python or Bash</li><li>Familiar with CI/CD processes using tools like Git and Jenkins</li><li>Experience with data modeling, enrichment logic, and system integration</li><li>Advantage: familiarity with log schema standards (e.g., ECS, CIM)</li><li>Ability to work independently and deliver production-ready, scalable solutions – Mandatory</li></ul><br>