Overview
Responsibilities
Below are some of the responsibilities of a big data engineer:
Design the architecture of our big data platform
Perform and oversee tasks such as writing scripts, calling APIs, web scraping, and writing SQL queries
Design and implement data stores that support the scalable processing and storage of our high-frequency data
Maintain our data pipeline
Customize and oversee integration tools, warehouses, databases, and analytical systems
Configure and provide availability for data-access tools used by all data scientists
Job Qualifications and Skill Sets
Below are the qualifications expected of a big data engineer:
3 to 5 years of relevant data engineering experience
Bachelor’s degree or higher in computer science, data science, or a related field
Hands-on experience with data cleaning, visualization, and reporting
At least 2 years of relevant experience with real-time data stream platforms such as Kafka and Spark Streaming
Experience working in an agile environment
Familiarity with the Hadoop ecosystem
Experience with platforms such as MapReduce, Apache Cassandra, Hive, Presto, and HBase
Excellent analytical and problem-solving skills
Excellent communication and interpersonal skills
About Arwin Global Solutions
Arwin Global Solutions is driving Managed IT Services, Systems mix, Technology benefits and counselling organization. Throughout the long term, Arwin Global Solutions has procured the standing as an exceptional firm in a serious, unique commercial center.
