Welcome to my portfolio! I'm Pritpal Singh, a Senior Data Engineer with over 8+ years of experience in building data-intensive applications and solving complex scalability challenges. My expertise lies in collecting, sorting, and merging data in the B2B field. I have a track record of success in constructing robust data pipelines and integrating millions of raw records with business data.
I have led migrations from MongoDB to Snowflake, utilizing AWS Glue, s3, and Snowpipe, resulting in improved data analysis performance. Additionally, I have designed and implemented real-time data pipelines using technologies such as Airflow, Kafka, Pyspark, and AWS Glue.
These pipelines seamlessly integrate semi-structured data from multiple sources, enabling efficient data processing.Throughout my career, I have managed teams and implemented algorithms to consolidate and match disparate business data sources. I have provided clients with optimal technical solutions to achieve their business goals and resolved performance and scalability issues with large-scale distributed crawling projects.
In addition to my data engineering skills, I have experience in web scraping using frameworks like Scrapy. I have also utilized visualization tools like leaflet.js to present algorithms and insights in a user-friendly manner. My expertise spans a wide range of technologies, including Python, Django, Django Rest Framework, Redis, MongoDB, PostgreSQL, AWS, Selenium, Pandas, Data Science, and Pyspark.
I hold a Bachelor's degree in Computer Application and a Master's degree in Computer Application. My education, combined with my industry experience, has equipped me with a strong foundation in computer science and data engineering.
I invite you to explore my portfolio to learn more about my projects and achievements in the field of data engineering. Let's connect and discuss how I can contribute to your data-driven projects. Feel free to reach out to me at click here or connect with me on LinkedIn. I look forward to connecting with you and exploring potential collaborations.