Writing dummy snippets of code to read, manipulate, and build a simple ML model with PySpark.

Given a set of documents and the minimum required similarity threshold find the number of document pairs that exceed the threshold

This notebook contains detailed code for spark and machine learning and databricks

A laboratory to carry out experiments with PySpark

Trying best case apache spark working environment for robust data pipelines

An academic project carried out for the Distributed Data Analysis and Mining course (a. y. 2022/2023)

Treat Spark like pandas.

The current assignment is to write the python scripts for Apache Spark. The tasks are divided into three parts as below: WordCount-To count the occurrences of words in a book on a per-book basis and compare the results with those of Assignment1. pyspark.ml. feature- To count the tf-idf values for the unigram and bigrams using the pyspark.ml.feat…

Pyspark Codes for Machine Learning and Big Data

Data Visualization and Prediction for TripAdvisor's dataset at Brown-Datathon event

Machine Learning Task implemented in PySpark to parallelise K-Fold Cross Validation

ML examples with pyspark

Course work for Big Data Analytics with Spark

Recommendation engine using Apache Spark (PySpark) and Python using network theory

A UDF to evaluate Spark-MLlib classification model using PySpark

Spark Assignment

PySpark with pytest in dokcer

Kaggle's house prices competition