Big Data Modeling, MapReduce, Spark, PySpark @ Santa Clara University

big-data apache-spark algorithms glossary data-transformation pyspark data-engineering partitioning-algorithms monoid data-analysis mapreduce spark-dataframes data-algorithms data-partition mapreduce-algorithm santa-clara-university spark-rdd mapreduce-python pyspark-algorithms-book ai-llm

• Updated Nov 12, 2026
• HTML

《大数据挖掘技术》@复旦 课程项目，试图从搜狗实验室用户查询日志数据（2008）中找出搜索记录中有较高支持度关键词的频繁二项集。在实现层面上，我搭建了一个由五台服务器组成的微型 Hadoop 集群，并且用 Python 实现了 Parallel FP-Growth 算法中的三个 MapReduce 过程。

hadoop mapreduce fp-growth mapreduce-python

• Updated Mar 29, 2021
• Python

基于Item-based CF和XGBRegressor完成的用户对商品的推荐系统

collaborative-filtering pyspark mapreduce-python

• Updated Jul 6, 2021
• Python

KMeans, Cure and Canpoy algorithms are demonstrated using Pyspark.

machine-learning big-data spark apache-spark clustering mapreduce kmeans clustering-algorithm cure canopy big-data-analytics spark-cluster mapreduce-python

• Updated May 19, 2021
• Jupyter Notebook

Modified from big-data-europe/docker-hadoop

docker hadoop mapreduce-python

• Updated Feb 14, 2024
• Python

Using hadoop to utilize data from an automobile tracking platform that tracks the history of important incidents after the initial sale of a new vehicle.

python hadoop virtualbox hortonworks-hdp mapreduce-python

• Updated Feb 19, 2022
• Python

A MapReduce framework implemented from scratch to perform K-mean clustering

distributed-systems consensus mapreduce consensus-algorithm mapreduce-algorithm mapreduce-python

• Updated Jun 14, 2024
• Python

Lambda to start EMR and run a map reduce job

aws aws-lambda aws-emr hadoop-mapreduce aws-emr-clusters mapreduce-python

• Updated Aug 16, 2019
• Python

This repository have codes that extracts meaningful information from News headline data-set.

python hadoop hadoop-mapreduce news-dataset mapreduce-python

• Updated Apr 28, 2019
• Python

A REST-based service that translates the SQL query into MapReduce and Spark jobs. It runs these jobs and provides the JSON object. SQL to MapReduce and Spark translator.

sql spark python3 pyspark mapreduce hadoop-mapreduce hadoop-streaming mapreduce-python sqltomapreduce sqltospark

• Updated Sep 30, 2020
• Python

A repository containing the source codes for the assignments done as a part of the Big Data course (UE18CS322) at PES University.

big-data spark hadoop analysis spark-streaming mapreduce-python

• Updated Jan 14, 2021
• Python

Pulled 10GB ofYelp Business data through the terminal via Kaggle API. The data was then pushed to and AWS S3 Bucket bucket for storage and analyzed on a Elastic MapReduce Cluster on a Jupyter Notebook using PySpark

spark cluster aws-s3 jupyter-notebook bucket pyspark elasticsearch-cluster mapreduce-python

• Updated Aug 21, 2023
• Jupyter Notebook

⚡️공개용 맵리듀스 플랫폼인 Spark를 사용하여 데이터마이닝을 해보자⚡️

spark bigdata jupyter-notebook pyspark mapreduce mapreduce-python dataminig

• Updated Apr 8, 2026
• Jupyter Notebook

A distributed map-reduce implemented by Python 3 and gRPC

distributed-computing grpc mapreduce mapreduce-python

• Updated Mar 27, 2021
• Python

Market basket analysis of finding frequent itemsets using SON algorithm in Spark

python data-mining spark apriori-son mapreduce-python

• Updated Oct 6, 2018
• Python

MapReduce is a programming model and an associated implementation for processing and generating large data sets. Users specify a map function that processes a key/value pair to generate a set of intermediate key/value pairs, and a reduce function that merges all intermediate values associated with the same intermediate key. Many real-world tasks…

gcp inverted-index wordcount memcached-server mapreduce-python

• Updated Apr 24, 2024
• Python

Lightweight and extensible library to execute MapReduce-like jobs in Python

python map command multiprocessing parallel-computing python3 reduce mapreduce builder-pattern mapreduce-jobs joshua-bloch multiprocess mapreduce-algorithm mapreduce-python

• Updated Jan 2, 2025
• Python

Performing Map reduce to get the page rank on the WDC data.

hadoop mapreduce-python

• Updated Nov 24, 2023
• Python

Multiprocessing can be an effective way to speed up a time-consuming workflow via parallelization. This article illustrates how multiprocessing can be utilized in a concise way when implementing MapReduce-like workflows.

python multiprocessing map-reduce parallel-python python-multiprocessing mapreduce-python python-articles python-introduction

• Updated Oct 22, 2020
• Jupyter Notebook

Programs for MapReduce written in java with least complexity!

java hadoop java-8 mapreduce hadoop-mapreduce mapreduce-java mapreduce-python

• Updated Feb 3, 2024
• Java