About
Passionate data enthusiast.
Data Consultant & Data Engineer
- Academic Website: Click here
- Phone: +49-XXXX-XXX-XXXX
- Degree: Ph.D. in Astrophysics (Cosmology)
- Current City: Mainz, Germany
Resume
Summary
Profile
Astrophysicist turned 'Data Consultant | Data Engineer' with over 5 years of postdoctoral research experience from leading international organizations. Currently working as a Data Engineer, specializing in advanced Python, web scraping, machine learning, deep learning, and data engineering. Skilled in building ETL pipelines, data streaming, and creating APIs. Proficient in a wide range of technologies including SQL, MongoDB, Elasticsearch, AWS, Snowflake, Apache NiFi, Docker, and Kubernetes. Passionate about leveraging data analytics and engineering to drive insightful and impactful solutions.
- Mainz, Rheinland-Pfalz, Germany
- +49-XXXX-XXX-XXXX
Diploma Courses
Bootcamp: Data Engineering
Dec 2023 - May 2024
Learning objectives: Completed a comprehensive Data Engineering Bootcamp, mastering essential technologies and tools.
- Create, customize, and manage ETLs to execute data projects
- Databases, data warehousing, data lake
- Python for advanced databases (SQL, NoSQL), Big Data (Hadoop, Spark), Git, GitHub/Gitlab, CI/CD (Jenkins, Github action), API (Flask, FastAPI), Airflow, Docker, Kubernetes, Web scraping, Docker containerization, Project management
Education
Ph.D. in Astrophysics (Cosmology)
2017
- Ph.D. thesis title: Origin and Dynamics of the Primordial Magnetic Field in a Parity-Violating Plasma
- Focused on comprehending magnetogenesis in the early universe during phase transitions and inflation
- Investigated the origin of primordial gravitational waves within a magnetized plasma
Master of Science (M.Sc.) in Physics
D.D.U. Gorakhpur University, Gorakhpur, India
Major on electronics.
Bachelor of Science (B.Sc.) in Physics, Mathematics, Electronics
D.D.U. Gorakhpur University, Gorakhpur, India
Major on electronics.
Professional Experience
EPS-SG system IVV Engineer at EUMETSAT, Darmstadt, Germany
October 2024 - Present (Permanent)
EUMETSAT | Sopra Steria | CS Group | HE-Space - Germany, Darmstadt, Deutschland
- Working as Integration Validation and Verification (IV&V) Engineer at EUMETSAT, Darmstadt.
- Involved with the team working on EUMETSAT Polar System – Second Generation (EPS-SG) and other interesting projects.
- Main tasks are Data processing from L0 to L1 and L2 in the TGPS systems and monitoring data flow between various interfaces for anomalies.
- Test procedure and executions for the various processors. Managed project complete test cycle on jira and configured it to the Xray test plugins for better test management.
- Proper documentations of various issues and procedures are one of the other tasks.
Data Consultant | Data Engineer
January 2023 - September 2024 (Full-time)
Mainz, Rheinland-Pfalz, Germany
- Created and deployed a scalable API using FastAPI, Elasticsearch, and Docker for analyzing diverse job market data sources. Complete pipeline helps in seamless data extraction, transformation, loading, and execution of predefined analytical tasks.
- Developed and deployed a dynamic web application using Python Flask, showcasing proficiency in Flask fundamentals and template utilization.
- Engineered data analytics solutions for Spotify's global song dataset, ensuring secure data handling and achieving streamlined project management.
- Architected end-to-end ETL pipelines leveraging Docker, Jupyter Notebook, Apache NiFi, and Snowflake, facilitating real-time data flow to AWS S3.
- Built data warehouses using Snowflake and created data pipelines using Apache Airflow on AWS.
- Conducted extensive risk analysis using Monte Carlo simulations for market and portfolio risk management, quantifying potential losses under various scenarios.
- Published blogs on science, finance, data science, and analytics, showcasing expertise and thought leadership.
Visiting Scientist
June 2022 - July 2022 (Stipendium)
Max-Planck-Institut für Astrophysik, Garching bei München, Germany
Postdoctoral Research Scientist
July 2020 - December 2020 (Full-time)
Hangzhou Institute for Advanced Studies, Hangzhou, Zhejiang, China
- Studied the Effects of gravitational waves on electromagnetic fields.
- Explored gravitational wave generation mechanisms in hot dense neutrino plasmas and their potential explanation for the NANOGrav signal, including constraints on magnetic field strength, which results in one publication in a respected international journal.
D.S. Kothari Postdoctoral Research Scientist
June 2019 – June 2020, January 2021 - May 2022 (Full-time)
Department of Physics and Astrophysics, University of Delhi, India
- Designed models and formulated algorithms: Created Python code to dissect the global temperature and ionization history of the universe. Evaluated the impacts of exotic energy sources like dark matter annihilation or decay.
- Conducted detailed mathematical calculations and solved complex equations on gravitational collapse of hot gaseous plasma in magnetic fields using Python.
Student Teaching Assistant
July 2019 – December 2019
Department of Physics and Astrophysics, University of Delhi, India
- Worked as a teaching assistant to the Masters student (M.Sc.) Mathematical Physics.
Postdoctoral Research Fellow
June 2017 – June 2019 (Full-time)
Physical Research Laboratory, Ahmedabad, India
- Innovative Model Building: Studied primordial field generation in the early universe, contributing to a deeper understanding of cosmic magnetism.
- Skilled in overseeing High-Performance Computing (HPC) systems for extensive computational work. Expertise in automating and optimizing data-driven operations in intricate networked environments.
Portfolio
Welcome to my portfolio, a hub of knowledge and expertise spanning various domains in data science, programming, and scientific research. Whether you're a budding data scientist, a seasoned developer, or a researcher seeking valuable insights, these resources are here to empower your growth and expand your horizons (related codes are Available at my Github repository).
- All
- Data analytics
- Machine-Learning
- Monte Carlo Simulation
- Academic codes
- Data engineering
- Remote sensing
Skills and Tools
Here are some of the skills I specialize in:
- Python: (Numpy, Pandas, Seaborn, Matplotlib, Statsmodels, Scipy, Plotly, scikit-learn, xarray, Satpy, GeoPandas)
- Machine Learning: (Supervised Learning: Linear, Polynomial & Logistic Regression, Decision Trees, K-nearest Neighbors (KNN), Support Vector Machines (SVMs), Random Forest, Naive Bayes, Gradient Descent), (Unsupervised Learning: Principal Component Analysis (PCA)), ARIMA, TensorFlow, Maximum Likelihood Estimation
- Artificial Inteligence: (Natural Language Processing (NLP))
- Version Control: (Git, GitHub, GitLab)
- Database Management: (SQL, BigQuery, MySQL, PostgreSQL, Elasticsearch, MongoDB, Luna Data Modeler)
- Data Engineering: (Data Warehouse, Data Lake, Snowflake, Apache Airflow, Kafka, FastAPI, Flask, CI/CD pipeline, Dash, Unit Test, ETL/ELT Processes, Atlassian Tools (Jira, Confluence, Trello), Amazon Web Services (AWS), Redshift)
- Scripting: (Bash, Shell)
- Web Development: (HTML & CSS)
- Software and Tools: (VSCode, API Integrations, Virtual Machine, Docker Containerization, Bitbucket, Mathematica)
- Operating Systems: (Linux Environments, Windows, Mac-OS)
- Simulation: (Monte Carlo Simulation, OpenMPI)
- Dashboard Tools: (MS Excel, Power BI, Tableau, Looker Studio)
- Documentation: (LaTeX, MS Word, Mac Pages)
Other Skills
Here are some additional skills and tools I specialize in:
- Level-0 to 2 Meteorological Datasets
- Remote Sensing
- Time Series Analysis
- ETL (Extract, Transform, Load) Process
- Anomaly Investigation
- Statistical Analysis
- Data Mining
- Data Modeling
- Predictive Analytics
- Quick Learner
- Data Pipeline on Airflow
- Cross-Functional Collaboration
- Highly Organized
- Problem-Solving Abilities
- Communication Skills
- Detail Oriented
- Fluent in English
- Quality Control and Validation
Technical Skills
With my years of work in the research field, I have acquired the following technical skills:
Python
Machine Learning
Data Engineering
NLP
SQL
Visualization
HTML/CSS
Documentation
Languages
English Fluent
Hindi (Native)
German (A2)
Bhojpuri (Native)
Chinese (Beginner)
Awards / Fellowship / Recognition
- D. S. Kothari Post Doctoral Fellowships (DSKPDF) in Sciences, UNIVERSITY GRANTS COMMISSION (UGC)
- Year: April 2019
- Grant number: (BSR) PH/18-19/0070
- I was one of only 15 candidates selected nationwide for the April 2019 cycle.
- Subject: Astrophysics
- CSIR-Junior Research Fellowship (CSIR-JRF)
- Year: June 2011
- Qualified for the National Eligibility Test (NET) for Lectureship (June, 2011),Conduct by CSIR (Council Of Scientific And Industrial Research), & University Grant Commission (UGC) under Ministry of Human Resource Development Organization India for doctorate fellowship.
- Subject: Physics
- Rank:All India Rank 33 (CSIR-JRF).
- Graduate Aptitude Test (GATE)
- Year: March 2011
- Qualified for the Graduate Aptitude Test in Engineering (GATE), Conducted by Indian Institutes of Technology (IITs) and Indian Institute of Science (IISc) on behalf of the National Coordination Board – GATE, Department of Higher Education, Ministry of Education (MoE), Government of India.
- Subject: Physics
- Percentile: 98.39
- Rank: All India Rank 107.
- Fellowship for the Doctoral Studies & Research
- Year: July, 2011 – Jun, 2016
- Physical Research Laboratory Ahmedabad, India, Department of Space, Government of India
- Subject: Theoretical Physics / Cosmology / Astrophysics
- Fellowship for the Doctoral Studies & Research
- Year: June, 2011
- The Institute for Plasma Research (IPR ), Gandhinagar, India, Department of Atomic Energy (DAE), Government of India
- Research area offered: ITER-India program
Professional Courses & Certification
- Operational Satellite Oceanography Workshop
- Visualise CoastWatch data, Use command line tools to perform data extractions from CoastWatch products.
- Access data operationally through the EUMETSAT Data Store APIs and EUMDAC client.
- Conduct batch processing using SNAP and supporting Jupyter notebooks.
- Extract and analyse in situ matchups with Sentinel-3 ocean colour data using ThoMaS - a Tool to generate Matchups of OC products with Sentinel-3/OLCI.
- Customized Jira configurations for project-specific needs, improving team productivity.
- Work with GOCI-II data.
- EUMETSAT Data Access Services & European Weather Cloud
- Completed an extensive course on remote sensing provided by EUMETSAT, gaining a strong foundation in data access and retrieval, the fundamentals of remote sensing, and Python data analysis of multidimensional meteorological datasets.
- Developed skills in using remote sensing data to analyze and monitor environmental phenomena such as climate change, weather patterns, and natural disasters.
- Learned to apply Python programming language to process and analyze remote sensing data, extracting valuable insights and information.
- Gained experience in working with a variety of remote sensing datasets, including satellite imagery and ground-based data.
- Data Warehouse for Data Engineering with Snowflake
- Mastered fundamentals of Data Warehouses.
- Gained in-depth knowledge of Dimension Modelling, including E-Commerce Dimension Modelling.
- Learned about Slowly Changing Dimension techniques.
- Acquired skills in Extract Transform Load (ETL) processes.
- Completed a project on Spotify Data Pipeline using Snowflake, AWS, and Python.
- Implemented Real-Time Data Streaming using AWS, Snowflake.
- Developed expertise in creating pipelines using Apache Airflow.
- Cloud Platform, Secure a cloud-based application at Verizon (virtual simulation)
- Completed a job simulation involving building a hypothetical new VPN product for Verizon’s Cloud Computing team.
- Used command line Python to test whether Verizon’s VPN met the cloud-native traits, i.e. redundancy, resiliency, and least privilege.
- Researched approaches to achieve application security and communicated insights in a PowerPoint Presentation.
- JPMorgan Chase Investment Banking Virtual Experience Program on Forage
- Identified an ideal M&A target for a client based on an assessment of their strategic and financial criteria.
- Constructed a DCF model to calculate the valuation of the M&A target and adjusted the model to account for a competitor bid and supply chain interruption.
- Created a 2-pager for the client containing a company profile and summary of the auction process.
- Tata Data Visualisation: Empowering Business with Effective Insights
- Proactively addressing vital business inquiries through tailored data visualization and interpretation for leaders.
- Proficiently selecting the most suitable visuals, such as charts and graphs, for effective communication of complex data.
- Specializing in crafting impactful visuals that align with business requirements, including expertise in data visualization, dashboard development, and data refinement.
- Skillfully conveying insights to diverse audiences and providing clear explanations of data's significance in various contexts.
- Machine Learning with Python: Supervised learning
- Regression models
- Classification model: K-nearest neighbor
- Data Analysis with Python: Zero to Pandas
- Numpy, Pandas, Seaborn, Matplotlib, Plotly, Statsmodel, Scipy, Sklearn
Nov 2023 – Dec 2023
Sep 2023 – Dec 2023
Sep 2023 – Sep 2023
Sep 2023
Aug 2023
Feb 2023 – Mar 2023
Oct 2022 – Dec 2022
My Orcid id
- Primordial Magnetic field and kinetic theory with Berry curvature, Jitesh R. Bhatt, Arun Kumar Pandey [arXiv:1503.01878 [astro-ph.CO]] Phys.Rev.D. 94, 043536
- Primordial Generation of magnetic field, Jitesh R. Bhatt, Arun Kumar Pandey [arXiv:1507.01795 [gr-qc]] Springer Proc.Phys. 174 (2016) 409-413
- Effect of background magnetic field on the normal modes of conformal dissipative chiral hydro and a novel mechanism for explaining pulsar kicks Arun Kumar Pandey, Manu George [arXiv:1609.01848 (astro-ph.CO)]
- Chiral Battery, scaling laws and magnetic fields, Sampurn Anad, Jitesh R. Bhatt & Arun Kumar Pandey [arXiv:1705.03683 (astro-ph.CO)] JCAP, JULY 2017
- Chiral Plasma Instability and Primordial Gravitational waves, Sampurn Anad, Jitesh R. Bhatt & Arun Kumar Pandey [arXiv:1801.00650 [astro-ph.CO] (2019)] Eur. Phys. J. C (2019) 79: 119.
- Baryon-Dark matter interaction in presence of magnetic fields in light of EDGES signal, Jitesh R Bhatt, Pravin Kumar Natwariya, Aleka C. Nayak, Arun Kumar Pandey [arXiv:1905.13486 [astro-ph.CO] (2019)], Eur. Phys. J. C 80, 334 (2020)
- Viscosity in cosmic fluids, Jitesh R Bhatt, Pravin Kumar Natwariya, Arun Kumar Pandey, arXiv:1907.03445 [astro-ph.CO] (2019) Eur. Phys. J. C 80 (2020) 8, 767
- Magnetic fields in a hot dense neutrino plasma and the Gravitational Waves Arun Kumar Pandey, Pravin Kumar Natwariya, Jitesh R Bhatt, arXiv:1911.05412 [astro-ph.CO] (2020), Phys. Rev. D 101, 023531 (2020)
- Implications of baryon-dark matter interaction on IGM temperature and tSZ effect with magnetic field, Arun Kumar Pandey, Sunil Malik, T. R. Seshadri, arXiv:2006.07901 [astro-ph.CO] (2020), Mon.Not.Roy.Astron.Soc. 500 (2020)
- Gravitational waves in neutrino plasma and NANOGrav signal, Arun Kumar Pandey arXiv:2011.05821 [astro-ph.CO] (2020) [Eur.Phys.J.C 81 (2021) 5, 399]
- Generating Seed magnetic field à la Chiral Biermann battery, Arun Kumar Pandey, Sampurn Anand (Phys. Rev. D. 104, 063508 (2021))
- Thermal SZ effect in a magnetized IGM dominated by interacting DM decay/annihilation during dark ages, Arun Kumar Pandey, Sunil Malik (2022) [arXiv:2204.08088]
- Spherical collpase model of a magnetized cloud, Arun Kumar Pandey (under preparation), 2024
Contact
You can contact me at my email: arunp77@gmail.com.
Location:
Mainz, Germany
Email:
arunp77@gmail.com
arunp77@live.in
arun-kumar.pandey@soprasteria.com
Call:
+49 XXXX-XXX-XXXX
