Learning

Learning from examples (Chapters 18)

class DataSet(examples=None, attrs=None, attr_names=None, target=-1, inputs=None, values=None, distance=<function mean_boolean_error>, name='', source='', exclude=())[source]

Bases: object

A data set for a machine learning problem. It has the following fields:

d.examples   A list of examples. Each one is a list of attribute values.
d.attrs      A list of integers to index into an example, so example[attr]
             gives a value. Normally the same as range(len(d.examples[0])).
d.attr_names Optional list of mnemonic names for corresponding attrs.
d.target     The attribute that a learning algorithm will try to predict.
             By default the final attribute.
d.inputs     The list of attrs without the target.
d.values     A list of lists: each sublist is the set of possible
             values for the corresponding attribute. If initially None,
             it is computed from the known examples by self.set_problem.
             If not None, an erroneous value raises ValueError.
d.distance   A function from a pair of examples to a non-negative number.
             Should be symmetric, etc. Defaults to mean_boolean_error
             since that can handle any field types.
d.name       Name of the data set (for output display only).
d.source     URL or other source where the data came from.
d.exclude    A list of attribute indexes to exclude from d.inputs. Elements
             of this list can either be integers (attrs) or attr_names.

Normally, you call the constructor and you’re done; then you just access fields like d.examples and d.target and d.inputs.

set_problem(target, inputs=None, exclude=())[source]

Set (or change) the target and/or inputs. This way, one DataSet can be used multiple ways. inputs, if specified, is a list of attributes, or specify exclude as a list of attributes to not use in inputs. Attributes can be -n .. n, or an attr_name. Also computes the list of possible values, if that wasn’t done yet.

check_me()[source]

Check that my fields make sense.

add_example(example)[source]

Add an example to the list of examples, checking it first.

check_example(example)[source]

Raise ValueError if example has any invalid values.

attr_num(attr)[source]

Returns the number used for attr, which can be a name, or -n .. n-1.

update_values()[source]

Recompute self.values (the list of distinct values per attribute) from the examples.

sanitize(example)[source]

Return a copy of example, with non-input attributes replaced by None.

classes_to_numbers(classes=None)[source]

Converts class names to numbers.

remove_examples(value='')[source]

Remove examples that contain given value.

split_values_by_classes()[source]

Split values into buckets according to their class.

find_means_and_deviations()[source]

Finds the means and standard deviations of self.dataset:

means     : a dictionary for each class/target. Holds a list of the means
            of the features for the class.
deviations: a dictionary for each class/target. Holds a list of the sample
            standard deviations of the features for the class.

parse_csv(input, delim=',')[source]

Input is a string consisting of lines, each line has comma-delimited fields. Convert this into a list of lists. Blank lines are skipped. Fields that look like numbers are converted to numbers. The delim defaults to ‘,’ but ‘t’ and None are also reasonable values.

err_ratio(predict, dataset, examples=None)[source]

Return the proportion of the examples that are NOT correctly predicted. verbose - 0: No output; 1: Output wrong; 2 (or greater): Output correct Accepts either a callable predictor or a learner object with a .predict method.

grade_learner(predict, tests)[source]

Grades the given learner based on how many tests it passes. tests is a list with each element in the form: (values, output). Accepts either a callable predictor or a learner object with a .predict method.

train_test_split(dataset, start=None, end=None, test_split=None)[source]

If you are giving ‘start’ and ‘end’ as parameters, then it will return the testing set from index ‘start’ to ‘end’ and the rest for training. If you give ‘test_split’ as a parameter then it will return test_split * 100% as the testing set and the rest as training set.

cross_validation_wrapper(learner, dataset, k=10, trials=1)[source]

[Figure 18.8] Return the optimal value of size having minimum error on validation set. errT: a training error array, indexed by size errV: a validation error array, indexed by size

cross_validation(learner, dataset, size=None, k=10, trials=1)[source]

Do k-fold cross_validate and return their mean. That is, keep out 1/k of the examples for testing on each of k runs. Shuffle the examples first; if trials > 1, average over several shuffles. Returns Training error, Validation error

leave_one_out(learner, dataset, size=None)[source]

Leave one out cross-validation over the dataset.

learning_curve(learner, dataset, trials=10, sizes=None)[source]

Return a list of (training-set size, mean accuracy) pairs, obtained by repeatedly cross-validating the learner on training sets of each given size.

PluralityLearner(dataset)[source]

A very dumb algorithm: always pick the result that was most popular in the training data. Makes a baseline for comparison.

class DecisionFork(attr, attr_name=None, default_child=None, branches=None)[source]

Bases: object

A fork of a decision tree holds an attribute to test, and a dict of branches, one for each of the attribute’s values.

add(val, subtree)[source]

Add a branch. If self.attr = val, go to the given subtree.

display(indent=0)[source]

Print this subtree, showing the tested attribute and each branch, indented by indent.

class DecisionLeaf(result)[source]

Bases: object

A leaf of a decision tree holds just a result.

display()[source]

Print the result stored at this leaf.

DecisionTreeLearner(dataset)[source]

[Figure 18.5] Learn a decision tree by recursively splitting the examples on the attribute with the highest information gain, until each branch is pure (or no attributes remain, falling back to the plurality class).

information_content(values)[source]

Number of bits to represent the probability distribution in values.

DecisionListLearner(dataset, max_test_size=None)[source]

[Figure 18.11] A decision list is a list of (test, outcome) pairs, where a test is a conjunction of (attribute, value) literals. An example is classified by the outcome of the first test it satisfies. Learning repeatedly finds a test that selects a non-empty subset of the remaining examples that all share a single outcome, appends (test, outcome), and removes those examples (Figure 18.11). Works on datasets with discrete attribute values.

NearestNeighborLearner(dataset, k=1)[source]

k-NearestNeighbor: the k nearest neighbors vote.

LinearLearner(dataset, learning_rate=0.01, epochs=100)[source]

[Section 18.6.3] Linear classifier with hard threshold.

LogisticLinearLeaner(dataset, learning_rate=0.01, epochs=100)[source]

[Section 18.6.4] Linear classifier with logistic regression.

NeuralNetLearner(dataset, hidden_layer_sizes=None, learning_rate=0.01, epochs=100, activation=<function sigmoid>)[source]

Layered feed-forward network. hidden_layer_sizes: List of number of hidden units per hidden layer learning_rate: Learning rate of gradient descent epochs: Number of passes over the dataset

BackPropagationLearner(dataset, net, learning_rate, epochs, activation=<function sigmoid>)[source]

[Figure 18.23] The back-propagation algorithm for multilayer networks.

PerceptronLearner(dataset, learning_rate=0.01, epochs=100)[source]

Logistic Regression, NO hidden layer

class NNUnit(activation=<function sigmoid>, weights=None, inputs=None)[source]

Bases: object

Single Unit of Multiple Layer Neural Network inputs: Incoming connections weights: Weights to incoming connections

network(input_units, hidden_layer_sizes, output_units, activation=<function sigmoid>)[source]

Create Directed Acyclic Network of given number layers. hidden_layers_sizes : List number of neuron units in each hidden layer excluding input and output layers

init_examples(examples, idx_i, idx_t, o_units)[source]

Split examples into input and target dicts keyed by example index.

Inputs are read from the attribute positions in idx_i and targets from position idx_t. When o_units > 1 each target is one-hot encoded over o_units units, otherwise it is wrapped in a single-element list. Returns the pair (inputs, targets).

find_max_node(nodes)[source]

Return the index of the node with the greatest value attribute.

class SVC(kernel=<function linear_kernel>, C=1.0, verbose=False)[source]

Bases: object

Support Vector Classifier trained in dual form by solving a quadratic programming problem; supports arbitrary kernels and a soft-margin penalty C.

fit(X, y)[source]

Trains the model by solving a quadratic programming problem. :param X: array of size [n_samples, n_features] holding the training samples :param y: array of size [n_samples] holding the class labels

solve_qp(X, y)[source]

Solves a quadratic programming problem. In QP formulation (dual): m variables, 2m+1 constraints (1 equation, 2m inequations). :param X: array of size [n_samples, n_features] holding the training samples :param y: array of size [n_samples] holding the class labels

predict_score(X)[source]

Predicts the score for a given example.

predict(X)[source]

Predicts the class of a given example.

class SVR(kernel=<function linear_kernel>, C=1.0, epsilon=0.1, verbose=False)[source]

Bases: object

Support Vector Regressor trained in dual form by solving a quadratic programming problem, using an epsilon-insensitive loss and penalty C.

fit(X, y)[source]

Trains the model by solving a quadratic programming problem. :param X: array of size [n_samples, n_features] holding the training samples :param y: array of size [n_samples] holding the class labels

solve_qp(X, y)[source]

Solves a quadratic programming problem. In QP formulation (dual): m variables, 2m+1 constraints (1 equation, 2m inequations). :param X: array of size [n_samples, n_features] holding the training samples :param y: array of size [n_samples] holding the class labels

predict(X)[source]

Predict the regression target value(s) for the samples X.

class MultiClassLearner(clf, decision_function='ovr')[source]

Bases: object

Wrap a binary classifier clf to handle multiple classes, using either the one-vs-rest (‘ovr’) or one-vs-one (‘ovo’) decision function.

fit(X, y)[source]

Trains n_class or n_class * (n_class - 1) / 2 classifiers according to the training method, ovr or ovo respectively. :param X: array of size [n_samples, n_features] holding the training samples :param y: array of size [n_samples] holding the class labels :return: array of classifiers

predict(X)[source]

Predicts the class of a given example according to the training method.

EnsembleLearner(learners)[source]

Given a list of learning algorithms, have them vote.

ada_boost(dataset, L, K)[source]

[Figure 18.34] AdaBoost: train K hypotheses with the weighted learner L, increasing the weight of misclassified examples after each round, and return their weighted-majority ensemble.

weighted_majority(predictors, weights)[source]

Return a predictor that takes a weighted vote.

weighted_mode(values, weights)[source]

Return the value with the greatest total weight.

RandomForest(dataset, n=5)[source]

An ensemble of Decision Trees trained using bagging and feature bagging.

WeightedLearner(unweighted_learner)[source]

[Page 749 footnote 14] Given a learner that takes just an unweighted dataset, return one that takes also a weight for each example.

replicated_dataset(dataset, weights, n=None)[source]

Copy dataset, replicating each example in proportion to its weight.

weighted_replicate(seq, weights, n)[source]

Return n selections from seq, with the count of each element of seq proportional to the corresponding weight (filling in fractions randomly).

accuracy_score(y_pred, y_true)[source]

Return the fraction of predictions in y_pred that match y_true.

r2_score(y_pred, y_true)[source]

Return the R^2 (coefficient of determination) of y_pred against y_true.

RestaurantDataSet(examples=None)[source]

[Figure 18.3] Build a DataSet of Restaurant waiting examples.

T(attr_name, branches)[source]

Build a DecisionFork testing the restaurant attribute attr_name, wrapping each non-fork child in a DecisionLeaf; a shorthand for writing decision trees by hand.

SyntheticRestaurant(n=20)[source]

Generate a DataSet with n examples.

Majority(k, n)[source]

Return a DataSet with n k-bit examples of the majority problem: k random bits followed by a 1 if more than half the bits are 1, else 0.

Parity(k, n, name='parity')[source]

Return a DataSet with n k-bit examples of the parity problem: k random bits followed by a 1 if an odd number of bits are 1, else 0.

Xor(n)[source]

Return a DataSet with n examples of 2-input xor.

ContinuousXor(n)[source]

2 inputs are chosen uniformly from (0.0 .. 2.0]; output is xor of ints.

gaussian_mixture_em(dataset, k, epsilon=0.0001, max_iterations=100)[source]

[Section 20.3] Unsupervised clustering with the Expectation-Maximization (EM) algorithm, fitting a mixture of k Gaussians to ‘dataset’ (a sequence of points). Each iteration performs two steps:

E-step: compute the responsibilities p_ij = P(C=i | x_j), the posterior
        probability that point x_j was generated by component i, which by
        Bayes' rule is proportional to P(x_j | C=i) * P(C=i).
M-step: re-estimate the weight, mean and covariance of each component as
        the responsibility-weighted statistics of the whole data set.

EM is guaranteed to increase the data log likelihood at each iteration; it is iterated until that improvement falls below ‘epsilon’ or ‘max_iterations’ is reached. Returns a dict with the fitted ‘weights’, ‘means’, ‘covariances’ and the final ‘responsibilities’.

naive_bayes_em(dataset, k, epsilon=0.0001, max_iterations=100)[source]

[Section 20.3] Learn the parameters of a Bayes net with a hidden variable via EM: a naive Bayes model with a hidden k-valued class (the ‘bags of candy’ example of Section 20.3.2). ‘dataset’ is a sequence of binary feature vectors; given the unobserved class, the features are independent Bernoulli variables. Each iteration performs two steps:

E-step: responsibilities r_ji = P(class=i | x_j), which by Bayes' rule and
        conditional independence is proportional to
        P(class=i) * prod_f P(x_jf | class=i).
M-step: re-estimate the class priors and every conditional probability
        P(feature_f = 1 | class=i) as the responsibility-weighted counts.

Returns a dict with the learned class ‘weights’, the ‘probabilities’ matrix (k x d, entry [i][f] = P(feature f = 1 | class i)) and the final ‘responsibilities’.

compare(algorithms=None, datasets=None, k=10, trials=1)[source]

Compare various learners on various datasets using cross-validation. Print results as a table.

Learning probabilistic models. (Chapters 20)

class CountingProbDist(observations=None, default=0)[source]

Bases: object

A probability distribution formed by observing and counting examples. If p is an instance of this class and o is an observed value, then there are 3 main operations: p.add(o) increments the count for observation o by 1. p.sample() returns a random element from the distribution. p[o] returns the probability for o (as in a regular ProbDist).

add(o)[source]

Add an observation o to the distribution.

smooth_for(o)[source]

Include o among the possible observations, whether or not it’s been observed yet.

top(n)[source]

Return (count, obs) tuples for the n most frequent observations.

sample()[source]

Return a random sample from the distribution.

NaiveBayesLearner(dataset, continuous=True, simple=False)[source]

Return a naive Bayes classifier for dataset, dispatching to the simple, continuous (Gaussian), or discrete variant according to simple and continuous.

NaiveBayesSimple(distribution)[source]

A simple naive bayes classifier that takes as input a dictionary of CountingProbDist objects and classifies items according to these distributions. The input dictionary is in the following form:

(ClassName, ClassProb): CountingProbDist

NaiveBayesDiscrete(dataset)[source]

Just count how many times each value of each input attribute occurs, conditional on the target value. Count the different target values too.

NaiveBayesContinuous(dataset)[source]

Count how many times each target value occurs. Also, find the means and deviations of input attribute values for each target value.

Deep learning. (Chapters 20)

element_wise_product(x, y)[source]

Element-wise product of x and y, recursing into nested iterables.

vector_add(a, b)[source]

Component-wise (recursive) addition of two vectors.

scalar_vector_product(x, y)[source]

Product of a scalar x and a (possibly nested) vector y, recursively.

class Node(weights=None, value=None)[source]

Bases: object

A single unit of a layer in a neural network :param weights: weights between parent nodes and current node :param value: value of current node

class Layer(size)[source]

Bases: object

A layer in a neural network based on a computational graph. :param size: number of units in the current layer

forward(inputs)[source]

Define the operation to get the output of this layer

class Activation[source]

Bases: object

Abstract base class for neural-network activation functions.

Subclasses implement function and its derivative; calling an instance applies the activation to its input.

function(x)[source]

Apply the activation function to input x.

derivative(x)[source]

Return the derivative of the activation function at x.

class Sigmoid[source]

Bases: Activation

Logistic sigmoid activation, 1 / (1 + e**-x).

function(x)[source]

Return the logistic sigmoid of x.

derivative(value)[source]

Return the sigmoid derivative given the layer output value.

class ReLU[source]

Bases: Activation

Rectified Linear Unit activation, max(0, x).

function(x)[source]

Return max(0, x).

derivative(value)[source]

Return the ReLU derivative (1 if value > 0 else 0).

class ELU(alpha=0.01)[source]

Bases: Activation

Exponential Linear Unit activation, with scale alpha for non-positive inputs.

function(x)[source]

Return x if positive else alpha * (e**x - 1).

derivative(value)[source]

Return the ELU derivative given the layer output value.

class LeakyReLU(alpha=0.01)[source]

Bases: Activation

Leaky ReLU activation, with small slope alpha for negative inputs.

function(x)[source]

Return max(x, alpha * x).

derivative(value)[source]

Return the Leaky ReLU derivative (1 if value > 0 else alpha).

class Tanh[source]

Bases: Activation

Hyperbolic tangent activation.

function(x)[source]

Return tanh(x).

derivative(value)[source]

Return the tanh derivative given the layer output value (1 - value**2).

class SoftMax[source]

Bases: Activation

Softmax activation that normalises a vector into a probability distribution.

function(x)[source]

Return the softmax of vector x (normalised exponentials).

derivative(x)[source]

Return a placeholder unit gradient for each element of x.

class SoftPlus[source]

Bases: Activation

SoftPlus activation, log(1 + e**x) (a smooth approximation of ReLU).

function(x)[source]

Return log(1 + e**x) for x.

derivative(x)[source]

Return the SoftPlus derivative at x (the logistic sigmoid).

class Linear[source]

Bases: Activation

Identity (linear) activation that returns its input unchanged.

function(x)[source]

Return x unchanged.

derivative(x)[source]

Return an all-ones gradient matching the shape of x.

class InputLayer(size=3)[source]

Bases: Layer

1D input layer. Layer size is the same as input vector size.

forward(inputs)[source]

Take each value of the inputs to each unit in the layer.

class OutputLayer(size=3)[source]

Bases: Layer

1D softmax output layer in 19.3.2.

forward(inputs, activation=<class 'aima.deep_learning.SoftMax'>)[source]

Apply activation (softmax by default) to inputs and store it in each node.

class DenseLayer(in_size=3, out_size=3, activation=<class 'aima.deep_learning.Sigmoid'>)[source]

Bases: Layer

1D dense layer in a neural network. :param in_size: (int) input vector size :param out_size: (int) output vector size :param activation: (Activation object) activation function

forward(inputs)[source]

Apply the activation to each unit’s weighted sum of inputs and return the outputs.

class ConvLayer1D(size=3, kernel_size=3)[source]

Bases: Layer

1D convolution layer of in neural network. :param kernel_size: convolution kernel size

forward(features)[source]

Convolve each input channel in features with its node kernel and return the outputs.

class MaxPoolingLayer1D(size=3, kernel_size=3)[source]

Bases: Layer

1D max pooling layer in a neural network. :param kernel_size: max pooling area size

forward(features)[source]

Apply 1D max pooling over each channel in features and return the pooled outputs.

class BatchNormalizationLayer(size, eps=0.001)[source]

Bases: Layer

Batch normalization layer.

forward(inputs)[source]

Normalise inputs by their mean and std, then scale and shift by the layer weights.

init_examples(examples, idx_i, idx_t, o_units)[source]

Init examples from dataset.examples.

stochastic_gradient_descent(dataset, net, loss, epochs=1000, l_rate=0.01, batch_size=1, verbose=False)[source]

Gradient descent algorithm to update the learnable parameters of a network. :return: the updated network

adam(dataset, net, loss, epochs=1000, rho=(0.9, 0.999), delta=1e-08, l_rate=0.001, batch_size=1, verbose=False)[source]

[Figure 19.6] Adam optimizer to update the learnable parameters of a network. Required parameters are similar to gradient descent. :return the updated network

BackPropagation(inputs, targets, theta, net, loss)[source]

The back-propagation algorithm for multilayer networks in only one epoch, to calculate gradients of theta. :param inputs: a batch of inputs in an array. Each input is an iterable object :param targets: a batch of targets in an array. Each target is an iterable object :param theta: parameters to be updated :param net: a list of predefined layer objects representing their linear sequence :param loss: a predefined loss function taking array of inputs and targets :return: gradients of theta, loss of the input batch

get_batch(examples, batch_size=1)[source]

Split examples into multiple batches

class NeuralNetworkLearner(dataset, hidden_layer_sizes, l_rate=0.01, epochs=1000, batch_size=10, optimizer=<function stochastic_gradient_descent>, loss=<function mean_squared_error_loss>, verbose=False, plot=False)[source]

Bases: object

Simple dense multilayer neural network. :param hidden_layer_sizes: size of hidden layers in the form of a list

fit(X, y)[source]

Train the network with the configured optimizer and loss, returning self.

predict(example)[source]

Forward-pass example through the trained net and return the index of the max output.

class PerceptronLearner(dataset, l_rate=0.01, epochs=1000, batch_size=10, optimizer=<function stochastic_gradient_descent>, loss=<function mean_squared_error_loss>, verbose=False, plot=False)[source]

Bases: object

Simple perceptron neural network.

fit(X, y)[source]

Train the perceptron with the configured optimizer and loss, returning self.

predict(example)[source]

Forward-pass example and return the index of the maximum output unit.

keras_dataset_loader(dataset, max_length=500)[source]

Helper function to load keras datasets. :param dataset: keras data set type :param max_length: max length of each input sequence

SimpleRNNLearner(train_data, val_data, epochs=2, verbose=False)[source]

RNN example for text sentimental analysis.

Parameters:

• train_data – a tuple of (training data, targets) Training data: ndarray taking training examples, while each example is coded by embedding Targets: ndarray taking targets of each example. Each target is mapped to an integer

• val_data – a tuple of (validation data, targets)

• epochs – number of epochs

• verbose – verbosity mode

Returns:

a keras model

AutoencoderLearner(inputs, encoding_size, epochs=200, verbose=False)[source]

Simple example of linear auto encoder learning producing the input itself. :param inputs: a batch of input data in np.ndarray type :param encoding_size: int, the size of encoding layer :param epochs: number of epochs :param verbose: verbosity mode :return: a keras model

Reinforcement Learning (Chapter 21)

class PassiveDUEAgent(pi, mdp)[source]

Bases: object

Passive (non-learning) agent that uses direct utility estimation on a given MDP and policy:

import sys
from aima.mdp import sequential_decision_environment
north = (0, 1)
south = (0,-1)
west = (-1, 0)
east = (1, 0)
policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, (0, 1): north, (2, 1): north,
          (3, 1): None, (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west,}
agent = PassiveDUEAgent(policy, sequential_decision_environment)
for i in range(200):
    run_single_trial(agent,sequential_decision_environment)
    agent.estimate_U()
agent.U[(0, 0)] > 0.2
True

estimate_U()[source]

Update utility estimates from the most recent completed trial by direct utility estimation: average the observed reward-to-go for each visited state, blend it with the running estimate, and reset the trial history. Must be called only once the MDP has reached a terminal state.

update_state(percept)[source]

To be overridden in most cases. The default case assumes the percept to be of type (state, reward)

class PassiveADPAgent(pi, mdp)[source]

Bases: object

[Figure 21.2] Passive (non-learning) agent that uses adaptive dynamic programming on a given MDP and policy:

import sys
from aima.mdp import sequential_decision_environment
north = (0, 1)
south = (0,-1)
west = (-1, 0)
east = (1, 0)
policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, (0, 1): north, (2, 1): north,
          (3, 1): None, (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west,}
agent = PassiveADPAgent(policy, sequential_decision_environment)
for i in range(100):
    run_single_trial(agent,sequential_decision_environment)

agent.U[(0, 0)] > 0.2
True
agent.U[(0, 1)] > 0.2
True

class ModelMDP(init, actlist, terminals, gamma, states)[source]

Bases: MDP

Class for implementing modified Version of input MDP with an editable transition model P and a custom function T.

T(s, a)[source]

Return a list of tuples with probabilities for states based on the learnt model P.

update_state(percept)[source]

To be overridden in most cases. The default case assumes the percept to be of type (state, reward).

class PassiveTDAgent(pi, mdp, alpha=None)[source]

Bases: object

[Figure 21.4] The abstract class for a Passive (non-learning) agent that uses temporal differences to learn utility estimates. Override update_state method to convert percept to state and reward. The mdp being provided should be an instance of a subclass of the MDP Class:

import sys
from aima.mdp import sequential_decision_environment
north = (0, 1)
south = (0,-1)
west = (-1, 0)
east = (1, 0)
policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, (0, 1): north, (2, 1): north,
          (3, 1): None, (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west,}
agent = PassiveTDAgent(policy, sequential_decision_environment, alpha=lambda n: 60./(59+n))
for i in range(200):
    run_single_trial(agent,sequential_decision_environment)

agent.U[(0, 0)] > 0.2
True
agent.U[(0, 1)] > 0.2
True

update_state(percept)[source]

To be overridden in most cases. The default case assumes the percept to be of type (state, reward).

class QLearningAgent(mdp, Ne, Rplus, alpha=None)[source]

Bases: object

[Figure 21.8] An exploratory Q-learning agent. It avoids having to learn the transition model because the Q-value of a state can be related directly to those of its neighbors:

import sys
from aima.mdp import sequential_decision_environment
north = (0, 1)
south = (0,-1)
west = (-1, 0)
east = (1, 0)
policy = {(0, 2): east, (1, 2): east, (2, 2): east, (3, 2): None, (0, 1): north, (2, 1): north,
          (3, 1): None, (0, 0): north, (1, 0): west, (2, 0): west, (3, 0): west,}
q_agent = QLearningAgent(sequential_decision_environment, Ne=5, Rplus=2, alpha=lambda n: 60./(59+n))
for i in range(200):
    run_single_trial(q_agent,sequential_decision_environment)

q_agent.Q[((0, 1), (0, 1))] >= -0.5
True
q_agent.Q[((1, 0), (0, -1))] <= 0.5
True

f(u, n)[source]

Exploration function. Returns fixed Rplus until agent has visited state, action a Ne number of times. Same as ADP agent in book.

actions_in_state(state)[source]

Return actions possible in given state. Useful for max and argmax.

update_state(percept)[source]

To be overridden in most cases. The default case assumes the percept to be of type (state, reward).

class SARSALearningAgent(mdp, Ne, Rplus, alpha=None)[source]

Bases: QLearningAgent

[Section 21.3] An on-policy temporal-difference control agent (SARSA: State-Action-Reward- State-Action). It is identical to the Q-learning agent except for the update rule: instead of bootstrapping on the maximum Q-value over next actions, SARSA bootstraps on the Q-value of the action a1 that its exploration policy will actually take in the next state. Being on-policy, SARSA learns the value of the policy it is following, exploration included, rather than that of the greedy policy.

run_single_trial(agent_program, mdp)[source]

Execute trial for given agent_program and mdp. mdp should be an instance of subclass of mdp.MDP

Perception (Chapter 24)

array_normalization(array, range_min, range_max)[source]

Normalize an array in the range of (range_min, range_max)

gradient_edge_detector(image)[source]

Image edge detection by calculating gradients in the image :param image: numpy ndarray or an iterable object :return: numpy ndarray, representing a gray scale image

gaussian_derivative_edge_detector(image)[source]

Image edge detector using derivative of gaussian kernels

hog(image, cell_size=8, bins=9, block_size=2)[source]

Histogram of Oriented Gradients (HOG) feature descriptor [#446].

Implements Dalal & Triggs (2005) – the descriptor referenced by AIMA Chapter 24 (Perception), for which the book gives no pseudocode. The steps are: gradient magnitude and unsigned (0-180 deg) orientation; for each cell_size x cell_size cell, a bins-bin orientation histogram weighted by gradient magnitude; L2 normalization over each block_size x block_size block of cells; the normalized blocks concatenated into one feature vector.

Source: N. Dalal & B. Triggs, “Histograms of Oriented Gradients for Human Detection”, CVPR 2005.

laplacian_edge_detector(image)[source]

Extract image edge with laplacian filter

show_edges(edges)[source]

helper function to show edges picture

sum_squared_difference(pic1, pic2)[source]

SSD of two frames

gen_gray_scale_picture(size, level=3)[source]

Generate a picture with different gray scale levels

Parameters:

• size – size of generated picture

• level – the number of level of gray scales in the picture, range (0, 255) are equally divided by number of levels

:return image in numpy ndarray type

probability_contour_detection(image, discs, threshold=0)[source]

Detect edges/contours by applying a set of discs to an image :param image: an image in type of numpy ndarray :param discs: a set of discs/filters to apply to pixels of image :param threshold: threshold to tell whether the pixel at (x, y) is on an edge :return image showing edges in numpy ndarray type

group_contour_detection(image, cluster_num=2)[source]

Detecting contours in an image with k-means clustering :param image: an image in numpy ndarray type :param cluster_num: number of clusters in k-means

image_to_graph(image)[source]

Convert an image to an graph in adjacent matrix form

generate_edge_weight(image, v1, v2)[source]

Find edge weight between two vertices in an image :param image: image in numpy ndarray type :param v1, v2: verticles in the image in form of (x index, y index)

class Graph(image)[source]

Bases: object

Graph in adjacent matrix to represent an image

bfs(s, t, parent)[source]

Breadth first search to tell whether there is an edge between source and sink parent: a list to save the path between s and t

min_cut(source, sink)[source]

Find the minimum cut of the graph between source and sink

gen_discs(init_scale, scales=1)[source]

Generate a collection of disc pairs by splitting an round discs with different angles :param init_scale: the initial size of each half discs :param scales: scale number of each type of half discs, the scale size will be doubled each time :return: the collection of generated discs: [discs of scale1, discs of scale2…]

load_MINST(train_size, val_size, test_size)[source]

Load MINST dataset from keras

simple_convnet(size=3, num_classes=10)[source]

Simple convolutional network for digit recognition :param size: number of convolution layers :param num_classes: number of output classes :return a convolution network in keras model type

train_model(model)[source]

Train the simple convolution network

selective_search(image)[source]

Selective search for object detection :param image: str, the path of image or image in ndarray type with 3 channels :return list of bounding boxes, each element is in form of [x_min, y_min, x_max, y_max]

pool_rois(feature_map, rois, pooled_height, pooled_width)[source]

Applies ROI pooling for a single image and various ROIs :param feature_map: ndarray, in shape of (width, height, channel) :param rois: list of roi :param pooled_height: height of pooled area :param pooled_width: width of pooled area :return list of pooled features

pool_roi(feature_map, roi, pooled_height, pooled_width)[source]

Applies a single ROI pooling to a single image :param feature_map: ndarray, in shape of (width, height, channel) :param roi: region of interest, in form of [x_min_ratio, y_min_ratio, x_max_ratio, y_max_ratio] :return feature of pooling output, in shape of (pooled_width, pooled_height)