this is a library of machine learning algorithms implemented in python + numpy. this is an ongoing (self-) educational project, and focuses on well-commented, understandable code over efficiency. (although efficiency is considered to a degree, e.g. vector operations over iterations with for-loops) all code is intended to be well-commented, and stresses explanation of algorithm.

while this is an educational project meant to be coded generally from scratch, other resources are referenced when designing the code for validation of procedure and confirmation of algorithms. references are listed below.

algorithms are class-based and use fit() for training and predict() for estimation, following sklearn standards.

• tools.accuracy_score : calculates accuracy score for categorical data
• tools.batchGenerator : an iterator for minibatch gradient descent
• tools.one_hot : turn an integer-indexed list/array into a 2-D matrix of one-hot vectors
• tools.train_test_split : shuffles and splits data according to the train_size parameter

• linalg.Vector : do basic vector operations (based on the Udacity Linear Algebra Refresher course)
• linalg.Line : do basic line operations (based on the Udacity Linear Algebra Refresher course)
• tools.fliess_kappa : calculate Fliess' Kappa inter-rater agreement score for N x k matrix of data

• neural.MultiLayerPerceptron : multivariate MLP with stochastic minibatch gradient descent
• regression.ZeroRuleforRegression : baseline mean/median regression
• regression.LinearRegression : multivariate, with (stochastic/minibatch) gradient descent and l2 regularization
• regression.LogisticRegression : multivariate, with (stochastic/minibatch) gradient descent and l2 regularization

https://www.udacity.com/course/linear-algebra-refresher-course--ud953

http://college.cengage.com/mathematics/brase/understandable_statistics/7e/students/datasets/mlr/frames/frame.html

https://www.cs.toronto.edu/~frossard/post/linear_regression/

http://ozzieliu.com/2016/02/09/gradient-descent-tutorial/

https://machinelearningmastery.com/implement-linear-regression-stochastic-gradient-descent-scratch-python/

https://www.pyimagesearch.com/2016/10/17/stochastic-gradient-descent-sgd-with-python/

https://www.analyticsvidhya.com/blog/2016/01/complete-tutorial-ridge-lasso-regression-python/

https://beckernick.github.io/logistic-regression-from-scratch/

http://aimotion.blogspot.kr/2011/11/machine-learning-with-python-logistic.html

https://courses.cs.washington.edu/courses/cse599c1/13wi/slides/l2-regularization-online-perceptron.pdf

http://florianmuellerklein.github.io/nn/

https://rolisz.ro/2013/04/18/neural-networks-in-python/

https://www.analyticsvidhya.com/blog/2017/05/neural-network-from-scratch-in-python-and-r/

https://databoys.github.io/Feedforward/

http://peterroelants.github.io/posts/neural_network_implementation_part02/

https://www.youtube.com/watch?v=tIeHLnjs5U8