Tech stack:
-AWS: Commonly used technologies such as Lambda, S3, CloudWatch, Secrets Manager, Parameter Store, AWS Glue, Athena, IAM, EMR.
-Big Data: Apache Spark v3, Parquet format, Avro format
-Python Stack:
- Boto3 (AWS service integration)
- PySpark
- Pandas (Big Data with Spark and Dataframes)
- Unittest
- Pytest
- Moto (Unit test and mocking AWS services)
-DevOps: Terraform (AWS provider) & Jenkins (used for deployment)
-Git