Python machine learning framework
Keras models, tf.function tracing, custom training loops, and GPU training for university ML coursework. The top failure mode on multi-GPU labs is forgetting to set memory growth before the first tf.config call, the GPU OOM our tutors patch in one line. Verified CS graduates from Georgia Tech, Purdue, and BITS Pilani, starting at $20 per task, 12-hour average turnaround.
About
TensorFlow is an end-to-end machine learning framework that compiles Python or graph-based models into optimized execution across CPU, GPU, and TPU hardware. The 2.x release line made eager execution the default while keeping tf.function as the bridge to graph-mode performance, integrated Keras as the high-level API (tf.keras), and added the TensorFlow Data API (tf.data) for input pipelines. Students meet TensorFlow in CS229 (Stanford Machine Learning), CS231n (Stanford CNNs), CS224n (Stanford NLP), 6.S191 (MIT Deep Learning), and any course that grades model accuracy on held-out test sets.
The library splits into Keras (high-level model building with Sequential and Functional APIs), tf.GradientTape (for custom gradient computation outside the Keras fit loop), tf.data (input pipelines with map, batch, prefetch, shuffle), and tf.distribute (strategy classes for multi-GPU and TPU training). CSHH tutors deliver Keras models with explicit input shapes and named layers (for TensorBoard interpretability), training loops with @tf.function decoration where the speed-up justifies the tracing cost, data pipelines that overlap I/O and compute via prefetch(tf.data.AUTOTUNE), GPU memory growth set before the first GPU operation (tf.config.experimental.set_memory_growth to prevent the default behavior of grabbing all available VRAM), and TensorBoard logging for loss curves, weight histograms, and the computation graph.
Coursework
The canonical first deep learning project. Sequential model with Conv2D, MaxPooling2D, Flatten, Dense layers, sparse categorical cross-entropy loss, Adam optimizer, model.fit with validation_split=0.2, ModelCheckpoint to save the best weights, and EarlyStopping with patience=5 on val_loss. Tutors include confusion matrix on the test set and per-class accuracy breakdown.
CNN with batch normalization and dropout, tf.keras.preprocessing.image.ImageDataGenerator or the newer tf.keras.layers.RandomFlip plus RandomRotation for in-graph augmentation, learning rate schedule via tf.keras.callbacks.LearningRateScheduler, and tensorboard --logdir for training curves. Common in CS231n assignment 2.
Pre-trained EfficientNetB0 from tf.keras.applications with include_top=False, custom classification head, two-phase training (freeze base then fine-tune with small learning rate), and tf.data pipeline reading from a directory tree with image_dataset_from_directory. Tutors include the BatchNormalization training=False trick for fine-tuning.
tf.keras.layers.Embedding for word vectors, Bidirectional LSTM or GRU, masking for variable-length sequences, padded batches via tf.data padded_batch, and a Dense output layer with sigmoid activation. Tutors include text preprocessing with tf.keras.layers.TextVectorization which stays inside the graph for export.
Manual training step using tf.GradientTape() to compute gradients, optimizer.apply_gradients(zip(grads, model.trainable_variables)), tf.function decoration for graph-mode speed-up, custom metrics with tf.keras.metrics.Mean and Result(), and a tqdm progress bar over the data iterator. Required for GAN training, contrastive learning, and any setup that does not fit the standard fit loop.
Generator and Discriminator as separate Keras models, separate optimizers, custom training loop alternating between G and D updates, binary cross-entropy loss with label smoothing, and image generation callback that samples from the latent space every N epochs. Tutors include mode collapse diagnostics and minibatch standard deviation.
tf.distribute.MirroredStrategy() context manager wrapping model build and compile, batch size scaled up by number of replicas (global_batch_size = per_replica_batch * num_replicas), distributed dataset via strategy.experimental_distribute_dataset, and learning rate scaled by sqrt(num_replicas) per the linear scaling rule. Tutors include the memory-growth setup that prevents the per-process VRAM hoarding pitfall.
Debugging
import tensorflow as tf
# TF grabs all VRAM on first GPU op
x = tf.constant([1.0])
for gpu in tf.config.list_physical_devices("GPU"):
tf.config.experimental.set_memory_growth(gpu, True)
# RuntimeError: device already initializedimport tensorflow as tf
# growth set BEFORE any GPU op
for gpu in tf.config.list_physical_devices("GPU"):
tf.config.experimental.set_memory_growth(gpu, True)
x = tf.constant([1.0]) # safe@tf.function
def step(x, batch_size):
# batch_size is a Python int -> new trace per value
return tf.reduce_mean(x) * batch_size@tf.function(input_signature=[
tf.TensorSpec(shape=[None, 784], dtype=tf.float32),
tf.TensorSpec(shape=[], dtype=tf.int32),
])
def step(x, batch_size):
return tf.reduce_mean(x) * tf.cast(batch_size, tf.float32)Either learning rate too high (gradient explosion), log of zero in custom loss (add epsilon: tf.math.log(x + 1e-7)), divide by zero in normalization, or training on int targets with categorical_crossentropy that expects one-hot. Diagnose by adding tf.debugging.check_numerics(loss, "loss is NaN") to the training step, then walking the computation backward to find the first NaN.
Dropout and BatchNormalization behave differently in train versus inference mode. If you wrote a custom training loop and forgot training=True on the model call during fit, BatchNorm uses its frozen moving stats instead of batch stats, training silently underperforms. Conversely, training=True at evaluation time leaks batch statistics into the prediction. Always pass training=True only inside the training step and training=False inside evaluation.
GPU sits at 30% utilization during training. The data pipeline is feeding too slowly. Fix by adding .prefetch(tf.data.AUTOTUNE) at the end of the pipeline (overlaps next-batch prep with current-batch compute), .cache() if the dataset fits in memory (or to disk for larger), .map(fn, num_parallel_calls=tf.data.AUTOTUNE) for parallel preprocessing, and inspecting with tf.data.experimental.bytes_produced_stats to see the throughput per stage.
Model expects (None, 224, 224, 3) but receives (None, 3, 224, 224). PyTorch uses channels-first by default; TensorFlow uses channels-last. Either transpose the input (tf.transpose(x, [0, 2, 3, 1])), or build the model with data_format="channels_first" which works only on GPU not CPU. Always confirm the data format of pre-trained weights you load.
tf.distribute.MirroredStrategy spawns one process per GPU. If memory growth is not set before the strategy is constructed, each process grabs its allocated GPU plus some scratch space on the others, leading to OOM. Set memory growth at the very top of the main module, before any imports that might initialize TensorFlow. For HuggingFace transformers, also disable the auto-tokenizer fast init that touches the GPU on import.
A model saved with TF 2.10 fails to load in TF 2.4 because newer ops (e.g., GroupNormalization) did not exist. Use model.save(filepath, save_format="h5") for the legacy HDF5 format which has wider compatibility, or pin the TensorFlow version in requirements.txt and rebuild the consumer environment. SavedModel format is forward-compatible but not always backward-compatible.
Code Examples
import tensorflow as tf
from tensorflow.keras import layers
# Set memory growth BEFORE any GPU op or you will OOM on shared boxes
for gpu in tf.config.list_physical_devices("GPU"):
tf.config.experimental.set_memory_growth(gpu, True)
model = tf.keras.Sequential([
layers.Conv2D(32, 3, activation="relu", input_shape=(28, 28, 1)),
layers.MaxPooling2D(),
layers.Conv2D(64, 3, activation="relu"),
layers.Flatten(),
layers.Dense(10),
])
model.compile(
optimizer="adam",
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=["accuracy"],
)
model.fit(
train_ds,
validation_data=val_ds,
epochs=10,
callbacks=[tf.keras.callbacks.EarlyStopping(patience=3, restore_best_weights=True)],
)train_ds = (
tf.data.Dataset.from_tensor_slices((x_train, y_train))
.shuffle(buffer_size=10_000)
.batch(128, drop_remainder=True)
.map(augment, num_parallel_calls=tf.data.AUTOTUNE)
.prefetch(tf.data.AUTOTUNE)
)Related
DATA 100 teaches the principles and techniques of data science across 14 weeks under Joseph Gonzalez, Narges Norouzi, and Lisa Yan (recent semesters), with co-development between EECS and the Division of Computing, Data Science, and Society. The course covers 6 modules: (1) Pandas and data wrangling, (2) exploratory data analysis (EDA) and visualization, (3) sampling and experimentation, (4) modeling and least-squares regression, (5) gradient descent and feature engineering, (6) classification with logistic regression, decision trees, and cross-validation. Languages and libraries: Python 3.11 with Pandas 2.x, NumPy 1.26, scikit-learn 1.4, matplotlib, seaborn, plotly, and statsmodels in select labs. The course assesses through 12 weekly Jupyter notebook homework assignments graded by Otter-Grader (an open-source autograder originally written for DATA 8 and extended for DATA 100), 2 projects (Project A1 housing-price regression, Project A2 spam-classification with logistic regression), a midterm at week 8, and a final at week 14. Lectures Monday and Wednesday at 5 PM in Wheeler Hall (or Pimentel for larger semesters), discussion Friday at varied times. Grading: 30 percent homework, 30 percent projects (15-15), 15 percent midterm, 20 percent final, 5 percent discussion attendance. The course is the second course in the Data Science major after DATA 8 (Foundations of Data Science) and is a prerequisite for DATA 101 (Data Engineering) and CS 189 (Machine Learning).
8 recurring assignments covered
Get help with DATA 100FAQ
Submit your TensorFlow assignment and get a working, commented solution within 12 hours from a verified CS graduate. Plagiarism-free, line-by-line annotated, with a reproducible test suite where the rubric allows it.
Submit TensorFlow Assignment