Data Engineer - BI Insights Group, Data Platform Department (DPD)

Job Description

Department Overview

The Data Platform Department (DPD) at Rakuten Group develops and maintains a comprehensive data platform, empowering over 70 Rakuten services with solutions for data ingestion, discovery, governance, analytics, and querying. We support data-driven decision-making across one of Japan's largest data ecosystems, providing the tools and infrastructure to support key domains such as Data Lakes, Data Warehouses, and Business Intelligence.

Position:

Position Details

Data Engineering & Infrastructure

• Hands-on design, development, testing, and implementation of scalable, cloud-based data warehouse solutions, optimized data pipelines, and robust data modeling
• Maintain a healthy data warehouse ecosystem through comprehensive infrastructure management (system logs, storage, compute resources, batch scheduling, etc.), ensuring seamless data integration and proactively addressing performance bottlenecks
• Internal Tooling: Adapt to and utilize Rakuten’s internal data pipeline orchestration framework, built on top of GCP-managed Airflow
• Communication: Collaborate with cross-functional teams to identify data requirements and align them with business objectives
• Documentation: Maintain comprehensive documentation, including platform architecture, data dictionaries, infrastructure configurations, operational processes and so on

Data Visualization & Analytics

• Collaborate with Stakeholders: Work closely with end-users to understand requirements, gather feedback, and deliver impactful, customized reports
• Solution Design: Propose and present visualization solutions to relevant stakeholders
• Data Modeling & Mart Design: Architect reporting-specific data marts to ensure high-performance dashboards
• Data Pipeline Management: Develop, maintain, and optimize pipelines, including DAG scheduling, data ingestion, and ETL/ELT processes to ensure data is "collected from" and "delivered to" efficiently and reliably
• Data Quality Assurance: Validate and ensure the accuracy, consistency, and reliability of all data used in visualizations
• Frontend Development: Design, build, and manage interactive reports using modern data visualization tools.

Mandatory Qualifications

• B.S. in Computer Science, or a related field, or equivalent education and experience
• 3+ years of end-to-end experience in data engineering (SQL) and operational experience in cloud-based or Linux systems
• Experience in developing and building robust data warehouse ecosystems using Google Cloud Platform (GCP – BQ, GCS), Python, and shell scripting
• Understanding of data warehouse/big data concepts, data modeling, batch-processing pipelines (ETL), and data integration techniques for managing large volumes of data
• Hands-on experience creating and leveraging data visualization platforms (e.g., DOMO, Tableau, MicroStrategy, PowerBI, etc.)
• Experience with Git

Desired Qualifications

Technical skills

• Experience with Atlassian tools (Confluence, Jira, BitBucket)
• Experience working on streaming data pipelines using tools such as Airflow, Apache Kafka, or Spark Streaming
• Experience managing systems with high Service Level Objectives/Agreements (SLOs/SLAs)

Soft skills

• Effective communication and project management skills
• Quick learner, highly adaptable, and committed to continuous learning
• Strong problem-solving and solution-creation mindset
• Detail-oriented and driven by excellence
• A "Can-Do" attitude and willingness to embrace new challenges
• Strong teamwork and collaboration capabilities

#engineer #DataEngineer #technologyplatformdiv