.png){kind=icon}
TensorLib
A zero-dependency open-source Machine Learning & Deep Learning library engineered completely from scratch in Python. Delivering PyTorch and Scikit-Learn style abstractions with dynamic backpropagation, neural operators, and classical ML algorithms.
Uses row-major contiguous list representation, stride index maps, dynamic topological sort, and automated reverse backpropagation.
From Raw Tensors to Autonomous Backpropagation
Built with modularity and educational transparency at its core.
To Be Mentioned
Core Tensor Engine
N-dimensional array storage with row-major memory layouts, stride indexing, broadcasting, and matrix multiplication.
Reverse Autograd
Dynamic computational graph tracking, topological sorting, and reverse backpropagation via
.backward().
Neural Network Layers
Dense (Linear), Conv2D, MaxPool2D, Sequential, Dropout, BatchNorm1d, Embeddings, LayerNorm, and RMSNorm.
Modern Optimizers
Complete optimizer suite including SGD with momentum, Adam, AdamW with decoupled weight decay, and RMSprop.
Classical ML Suite
Linear/Logistic Regression, Decision Trees, Random Forest, K-Means Clustering, K-NN, and PCA built on Tensor primitives.
Data & Utilities
TensorDataset, DataLoader mini-batching, StandardScaler, OneHotEncoder, graph visualizer
(render_graph), and model saving.
Quickstart Code Examples
1. Tensor Math & Computational Graph (Autograd)
Compute forward pass, render ASCII computational graph, and backpropagate gradients.
from tensorlib import Tensor
from tensorlib.utils import render_graph
# Create tensors with autograd enabled
x = Tensor([[1.0, 2.0], [3.0, 4.0]], requires_grad=True)
W = Tensor([[0.5, -0.5], [1.0, 2.0]], requires_grad=True)
# Forward matrix multiplication and loss calculation
y = x @ W
loss = (y ** 2).sum()
# Print ASCII computational graph
print(render_graph(loss))
# Reverse Backpropagation
loss.backward()
print("x.grad:", x.grad)
print("W.grad:", W.grad)2. Neural Network Model Training (Multi-Layer Perceptron)
Build custom sequential models with standard neural layers, optimizers, and losses.
from tensorlib import Tensor
from tensorlib.nn import Sequential, Linear, ReLU, CrossEntropyLoss
from tensorlib.optim import Adam
from tensorlib.data import TensorDataset, DataLoader
# Define dataset
X = Tensor([[1.0, 2.0], [1.5, 1.8], [5.0, 5.0], [6.0, 7.0]])
y = Tensor([0.0, 0.0, 1.0, 1.0])
dataset = TensorDataset(X, y)
loader = DataLoader(dataset, batch_size=2, shuffle=True)
# Build model
model = Sequential(
Linear(in_features=2, out_features=8),
ReLU(),
Linear(in_features=8, out_features=2)
)
optimizer = Adam(model.parameters(), lr=0.05)
criterion = CrossEntropyLoss()
# Training loop
for epoch in range(20):
for batch_X, batch_y in loader:
optimizer.zero_grad()
logits = model(batch_X)
loss = criterion(logits, batch_y)
loss.backward()
optimizer.step()3. Classical Machine Learning Suite
Scikit-Learn style interface operating seamlessly over Tensor primitives.
from tensorlib import Tensor
from tensorlib.ml import LogisticRegression, RandomForestClassifier, KMeans, PCA
X = Tensor([[1.0, 1.0], [1.5, 2.0], [6.0, 6.0], [7.0, 8.0]])
y = Tensor([0.0, 0.0, 1.0, 1.0])
# Logistic Regression
clf = LogisticRegression(lr=0.1, epochs=100).fit(X, y)
print("LogReg Predictions:", clf.predict(X).data)
# Random Forest Classifier
rf = RandomForestClassifier(n_estimators=5, max_depth=3).fit(X, y)
print("Random Forest Predictions:", rf.predict(X).data)
# K-Means Clustering
kmeans = KMeans(n_clusters=2, random_state=42).fit(X)
print("Cluster Assignments:", kmeans.predict(X).data)
# Principal Component Analysis
pca = PCA(n_components=1).fit(X)
X_reduced = pca.transform(X)
print("PCA Shape:", X_reduced.shape)API Reference: Core & Autograd
N-dimensional array with automated computational graph tracking and operator overloading.
Utility functions for instantiating initialized Tensors quickly.
API Reference: Neural Networks (tensorlib.nn)
| Category | Components | Description |
|---|---|---|
| Base Abstractions | Module, Parameter |
Base class tracking weights, sub-modules, and training/eval state. |
| Layers | Linear, Conv2D, MaxPool2D, Sequential,
Flatten, Dropout, BatchNorm1d, Embedding,
LayerNorm, RMSNorm |
Complete layer set for Feedforward, CNNs, and Transformer models. |
| Activations | ReLU, Sigmoid, Tanh, Softmax,
LeakyReLU, GELU, SiLU |
Differentiable non-linear transformation functions. |
| Loss Functions | MSELoss, L1Loss, BCEWithLogitsLoss,
CrossEntropyLoss |
Numerically stable loss functions for regression and multi-class target loss. |
| Transformer Engine | GPT, TransformerBlock, CausalSelfAttention,
FeedForward, GPTConfig |
Complete Causal Language Model supporting auto-regressive generation. |
API Reference: Optimizers (tensorlib.optim)
Stochastic Gradient Descent with momentum velocity and weight decay regularization.
Adaptive Moment Estimation computing bias-corrected 1st and 2nd raw moment estimates.
Adam with decoupled weight decay as introduced by Loshchilov & Hutter.
API Reference: Classical ML (tensorlib.ml)
| Algorithm | Class | Key Parameters |
|---|---|---|
| Linear & Logistic Regression | LinearRegression, LogisticRegression |
lr=0.01, epochs=1000 |
| Decision Tree & Forest | DecisionTreeClassifier, RandomForestClassifier |
max_depth=5, n_estimators=10 |
| Clustering | KMeans |
n_clusters=8, max_iters=300 |
| K-Nearest Neighbors | KNeighborsClassifier |
n_neighbors=5 |
| Dimensionality Reduction | PCA |
n_components=2 |
API Reference: Data & Evaluation Metrics
Dataset loaders, preprocessors, and tokenizers.
Standard performance evaluation metrics.
Repository Structure
Running Tests & Examples
Run the unit test suite built entirely with Python standard library unittest:
python -m unittest discover -s tests -p "test_*.py"Run example scripts:
python -m examples.01_tensor_autograd_basics
python -m examples.02_mlp_mnist_classification
python -m examples.03_classical_ml_regression_clustering
python -m examples.04_cnn_image_classifier
python -m examples.05_llm_gpt_trainingBuilt for research, learning, and developer clarity.
TensorLib is licensed under MIT. Feel free to inspect, extend, or contribute to the source repository.