Big Data Engineer Bootcamp

Big Data Engineer - Bootcamp

A unique training and certification program!

This master's program in Big Data Engineering, in collaboration with IBM, offers training in the competitive skills required for a rewarding career in data engineering. You will learn to master the comprehensive Hadoop data framework, leverage the functionality of Apache Spark with Python, simplify data pipelines with Apache Kafka, and use the open source database management tool MongoDB to store data in big data environments.

Key features

• Course and materials are in English
• 6-month online bootcamp and e-learning (at your own pace) can be completed faster!
• Industry-recognized certificates from IBM (for IBM courses) and Simplilearn
• Real-world projects that provide practical industry training
• 30+ in-demand skills
• 1 year of access to the learning platform and lesson recordings
• 35 hours of self-paced training
• 132 hours of instructor-led training
• Masterclasses, exclusive mentoring sessions, and hackathons by IBM

Program outcomes

Upon completion of the BIG DATA ENGINEER Bootcamp, you will be able to:

• Gain an in-depth understanding of the flexible and versatile frameworks of the Hadoop ecosystem, such as Pig, Hive, Impala, HBase, Sqoop, Flume, and Yarn
• Master tools and skills such as data model creation, database interfaces, advanced architecture, Spark, Scala, RDD, SparkSQL, Spark Streaming, Spark ML, GraphX, Sqoop, Flume, Pig, Hive, Impala, and Kafka architecture
• Understand how to model data, perform ingestion, replicate data, and shard data using the NoSQL database management system MongoDB
• Gain expertise in creating and maintaining analytics infrastructure and take responsibility for the development, distribution, maintenance, and monitoring of architecture components
• Gain insights into how you can improve your company's productivity by processing big data on platforms that can handle its volume, speed, variety, and reliability
• Learn how Kafka is used in real life, including its architecture and components, gain hands-on experience connecting Kafka to Spark, and work with Kafka Connect
• Understand how to use Amazon EMR to process data using Hadoop ecosystem tools

Who should enroll in this program?

A distinguished data engineer builds and maintains data structures and architectures for data acquisition, processing, and distribution for large-scale, data-intensive applications. It is a promising career for both new and experienced professionals with a passionate interest in data, including:

• IT professionals
• Banking and finance professionals
• Database administrators
• Beginners in the field of data engineering
• Students in UG/PG programs

Learning path – Big Data Engineer

1. Big Data for Data Engineering (1 hour)

This introductory course from IBM teaches you the basic concepts and terminology of big data and its practical applications in multiple industries. You will gain insights into how you can improve your company's productivity by processing large amounts of data and extracting valuable information.

Key learning objectives

• Understand what big data is, the sources of big data, and real-world examples
• Learn the crucial difference between big data and data science
• Become a master at using big data for operational analytics and better customer service
• Gain knowledge of the big data ecosystem and the Hadoop framework

Course outline

• Lesson 1 – What is big data?
• Lesson 2 – Big data: beyond the hype
• Lesson 3 – Big data and data science
• Lesson 4 – Use cases
• Lesson 5 – Big data processing

2. Big Data Hadoop and Spark Developer

AVC's Big Data Hadoop course helps you master big data and Hadoop ecosystem tools such as HDFS, YARN, MapReduce, Hive, Impala, Pig, HBase, Spark, Flume, Sqoop, and Hadoop Frameworks, including additional concepts in the big data processing lifecycle.

During this instructor-led online course, you will work on real-time projects in retail, tourism, finance, and other fields. This comprehensive data course prepares you for Cloudera's CCA175 Big Data certification.

Key learning objectives

• Learn to navigate the Hadoop ecosystem and understand how to optimize its use
• Import data with Sqoop, Flume, and Kafka
• Implement partitioning, bucketing, and indexing in Hive
• Work with RDDs in Apache Spark
• Process streaming data in real time
• Perform DataFrame operations in Spark using SQL queries
• Implement user-defined functions (UDFs) and user-defined attribute functions (UDAFs) in Spark

Course outline

• Lesson 1 – Introduction to Big Data and Hadoop
• Lesson 2 – Hadoop architecture, distributed storage (HDFS) and YARN
• Lesson 3 – Data import to Big Data systems and ETL
• Lesson 4 – Distributed processing, MapReduce framework and Pig
• Lesson 5 – Apache Hive
• Lesson 6 – NoSQL databases, HBase
• Lesson 7 – Fundamentals of functional programming and Scala
• Lesson 8 – Apache Spark – the next generation framework for big data
• Lesson 9 – Spark Core processing RDD
• Lesson 10 – Spark SQL processing of DataFrames
• Lesson 11 – Spark MLLib – modeling big data with Spark
• Lesson 12 – Streaming processing framework and Spark Streaming
• Lesson 13 – Spark GraphX

3. Pyspark training

The Pyspark training provides an in-depth overview of Apache Spark, the open source engine for processing large data sets, and how to integrate it with Python using the PySpark interface. This course shows you how to build and implement data-intensive applications while delving into high-performance machine learning. You will learn how to leverage Spark RDD, Spark SQL, Spark MLlib, Spark Streaming, HDFS, Sqoop, Flume, Spark GraphX, and Kafka.

Key learning objectives

• Understand how to leverage the functionality of Python when deploying it in the Spark ecosystem
• Master the Apache Spark architecture and how to configure a Python environment for Spark
• Learn about different techniques for collecting data, understand RDDs and how to compare them with DataFrames, how to read data from files and HDFS, and how to work with schemas
• Gain a comprehensive understanding of various tools included in the Spark ecosystem, such as Spark SQL, Spark MLLib, Sqoop, Kafka, Flume, and Spark Streaming
• Create and explore various APIs for working with Spark DataFrames and learn how to aggregate, transform, filter, and sort data with DataFrames

Course outline

• Lesson 01 – A brief introduction to Pyspark
• Lesson 02 – Resilient Distributed Datasets
• Lesson 03 – Resilient Distributed Datasets and actions
• Lesson 04 – DataFrames and transformations
• Lesson 05 – Data processing with Spark DataFrames

4. Apache Kafka

In this Apache Kafka certification course, you will learn the architecture, installation, configuration, and interfaces of Kafka open-source messaging. With this Kafka training, you will learn the basics of Apache ZooKeeper as a centralized service and develop skills to deploy Kafka for real-time messaging. The course is part of the Big Data Hadoop Architect Master's Program and is recommended for developers and analysts who want to deepen their knowledge.

Key learning objectives

• Describe the importance of big data
• Describe the basic concepts of Kafka
• Describe the architecture of Kafka
• Explain how to install and configure Kafka
• Explain how to use Kafka for real-time messaging.

Course outline

• Lesson 1 – Getting started with big data and Apache Kafka
• Lesson 2 – Kafka producer
• Lesson 3 – Kafka consumer
• Lesson 4 – Kafka operation and performance tuning
• Lesson 5 – Kafka cluster architecture and Kafka administration
• Lesson 6 – Kafka Monitoring and Schedule Registers
• Lesson 7 – Kafka Streams and Kafka Connect
• Lesson 8 – Integrating Kafka with Storm
• Lesson 9 – Kafka Integration with Spark and Flume
• Lesson 10 – Admin Client and Security in Kafka

5. MongoDB Developer and Administrator

Become an expert in MongoDB development and administration by deepening your knowledge of NoSQL and mastering skills in data modeling, data retrieval, search, sharding, and data replication. The course includes industry-based projects in e-learning and telecommunications. It is best suited for database administrators, software developers, system administrators, and analysts.

Key learning objectives

• Develop expertise in writing Java and NodeJS applications with MongoDB
• Master skills in replicating and sharding data in MongoDB to optimize read/write performance
• Perform installation, configuration, and maintenance of the MongoDB environment
• Gain practical experience in creating and managing different types of indexes in MongoDB for query execution
• Store unstructured data efficiently in MongoDB
• Develop skills for processing large amounts of data with MongoDB tools
• Become proficient in MongoDB configuration, backup methods, and monitoring and operational strategies
• Gain an in-depth understanding of DB Notes, Replica Set, and master-slave concepts.

Course outline

• Lesson 1 – Introduction to NoSQL databases
• Lesson 2 – MongoDB: A database for the modern web
• Lesson 3 – CRUD operations in MongoDB
• Lesson 4 – Indexing and aggregation
• Lesson 5 – Replication and sharding
• Lesson 6 – Developing Java and Node JS applications with MongoDB
• Lesson 7 – Managing MongoDB cluster operations

6. AWS Technical Essentials

This AWS Technical Essentials course teaches you how to navigate the AWS Management Console, understand AWS security measures, storage and database options, and gain expert knowledge of web services such as RDS and EBS. Designed in accordance with the latest AWS curriculum, the course will help you become proficient in identifying and effectively using AWS services.

Key learning objectives

• Understand the fundamental concepts of the AWS platform and cloud computing
• Identify AWS concepts, terminology, benefits, and deployment options to meet business requirements
• Identify deployment and networking options in AWS. Course outline

• Lesson 01 – Introduction to Cloud Computing
• Lesson 03 – Storage and Content Delivery
• Lesson 04 – Compute Services and Networking
• Lesson 05 – AWS Managed Services and Databases
• Lesson 06 – Deployment and Management

Lesson 02 – Introduction to AWS

7. AWS Big Data Certification Training

In this AWS Big Data Certification course, you will become familiar with the concepts of cloud computing and its deployment models, the Amazon Web Services cloud platform, Kinesis Analytics, AWS big data storage, processing, analysis, visualization, and security services, EMR, AWS Lambda and Glue, machine learning algorithms, and much more.

Key learning objectives

• Understand how to use Amazon EMR to process data using Hadoop ecosystem tools
• Understand how to use Amazon Kinesis for real-time big data processing
• Analyze and transform big data using Kinesis Streams
• Visualize data and perform queries using Amazon QuickSight.

Course

• Lesson 01 – AWS in Big Data – Introduction
• Lesson 02 – Collection
• Lesson 03 – Storage
• Lesson 04 – Processing I
• Lesson 05 – Processing II
• Lesson 06 – Analysis I
• Lesson 7 – Analysis II
• Lesson 8 – Visualization
• Lesson 9 – Security

Big Data Capstone

This Big Data Capstone project gives you the opportunity to apply the knowledge you have learned during the program. You will learn how to solve a real-world, industry-related big data problem through dedicated mentoring sessions. This project is the final step in the program and gives you the opportunity to showcase your big data expertise to future employers.

Optional course

• AWS Technical Essentials

The AWS Technical Essentials course teaches you how to navigate the AWS Management Console, understand AWS security measures, storage and database options, and gain expert knowledge of web services such as RDS and EBS. Designed in accordance with the latest AWS curriculum, the course helps you become proficient in identifying and effectively using AWS services.

• Java Certification Training

This advanced Java certification training is designed to guide you through Java concepts, from introductory techniques to advanced programming skills. This Java course also teaches you Core Java 8, operators, arrays, loops, methods, and constructors, while giving you hands-on experience with the JDBC and JUnit frameworks.

• Industry Masterclass – Data Engineering

Participate in an interactive online masterclass and gain insights into data engineering.

• SQL

This course provides you with all the information you need to successfully start working with SQL databases and use the database in your applications. Learn how to structure your database correctly, write effective SQL statements and clauses, and manage your SQL database for scalable growth.

• Industry Masterclass – Data Science

Join this interactive online industry masterclass to gain insights into advances in data science and AI techniques.

QUESTIONS

What is the value of a Master Certificate?

AVC's Bootcamp helps you master in-demand skills at a faster pace and increase your marketability. Whatever your career goals, whether you're a beginner or looking for skills development opportunities to change careers, AVC's master's programs will help accelerate those goals. These certificates are highly sought after.

Please note: We are not a university and do not issue university degrees. This Bootcamp Master's means that you have completed the entire program and gained all the essential knowledge about the subject, thereby mastering it fully. Each course/part of the program also results in a certificate.

What knowledge and experience are required for Bootcamp?

In general, no prior experience is required to participate in the program. The training starts at an introductory level and continues (step by step) to an expert level. However, it is always beneficial if you have basic knowledge or experience in the subject. Read more about your specific course details.

This program is self-paced, so you can learn at your own pace. You will start with a practical e-learning module. By the end of the program, you will have gained comprehensive knowledge and be able to demonstrate and apply it in a variety of practical tasks and projects.

How long does it take to complete Bootcamp?

This varies greatly from person to person. Some people complete the program fairly quickly (approx. 2-3 months), while others need more time. You have access to the program and eLearning for one year. If you spend 5-10 hours per week on the program, it will take you approx. 6 months to complete. Note: Some other master's programs take longer. This is an estimate.