Python data analysis library
DataFrame operations, groupby, merge, time series, and the SettingWithCopyWarning explained for university data science coursework. The top failure mode in data wrangling assignments is chained indexing that triggers SettingWithCopyWarning then silently fails to mutate, the bug our tutors patch with explicit .loc assignment. Verified CS graduates from Georgia Tech, Purdue, and BITS Pilani, starting at $20 per task, 12-hour average turnaround.
About
pandas is a Python data analysis library built on NumPy that provides the DataFrame and Series structures for labeled, tabular, and time-series data. The 2.x release line introduced PyArrow-backed string and extension dtypes, copy-on-write semantics (opt-in via pd.options.mode.copy_on_write=True, becoming default in 3.0), and significant performance improvements for groupby and merge operations. Students meet pandas in data science courses (CS109 Harvard, CS246 Stanford Mining Massive Datasets, 6.S897 MIT Machine Learning for Healthcare), in any course that grades exploratory data analysis or feature engineering, and in Kaggle-style competition assignments.
The library splits into IO (read_csv, read_parquet, read_sql, read_json, read_excel), data manipulation (loc and iloc indexing, query, assign, pipe), aggregation (groupby, agg, transform, apply), reshaping (pivot, pivot_table, melt, stack, unstack), merging (merge, join, concat), time series (date_range, resample, rolling, ewm, time zone localization and conversion), and visualization (DataFrame.plot built on matplotlib, with seaborn as the typical extension). CSHH tutors deliver DataFrame pipelines using method chains (.pipe for custom functions), .loc and .iloc for explicit indexing (avoiding the chained indexing that triggers SettingWithCopyWarning), groupby with agg passing a dict of column-to-function mappings for multi-output aggregation, merges that explicitly state how (inner, left, right, outer) and on (column name or index), and the .copy() pattern when a slice will be modified independently of the source.
Coursework
pd.read_csv with proper dtypes (avoid string for numeric columns to save memory), .info() and .describe() for the initial summary, .isna().sum() for missing-value audit, .value_counts() for categorical distributions, groupby aggregations for relationships, and matplotlib or seaborn for visualization. Common assignments in CS109 use the Titanic, Iris, or California Housing datasets.
pd.read_csv with parse_dates and index_col on a timestamp column, .asfreq("D") to enforce daily frequency with NaN for missing dates, .resample("M").mean() for monthly aggregation, .rolling(window=7).mean() for moving averages, .ewm(span=30).mean() for exponential weighted, and seasonal decomposition via statsmodels for trend and seasonality extraction.
One-hot encoding via pd.get_dummies or sklearn.preprocessing.OneHotEncoder, log transformation for skewed numeric features (np.log1p), categorical encoding (target encoding, leave-one-out, frequency encoding), interaction features (df["a_times_b"] = df["a"] * df["b"]), datetime features (year, month, day_of_week, is_weekend), and binning via pd.cut or pd.qcut.
pd.merge with how="inner" or "left", on=["customer_id", "date"] for composite keys, validate="one_to_many" to assert the expected cardinality, indicator=True to flag rows that did not match on either side, and suffixes=("_a", "_b") for overlapping column names. Tutors include the assertion that the merged row count matches expectation, catching silent join bugs early.
pd.pivot_table with index, columns, values, aggfunc (or dict of aggfuncs per column), margins=True for grand totals, fill_value=0 for missing cells, and dropna=False to keep the full grid. Tutors include the .melt counterpart that reverses pivot, useful for switching between wide and long formats.
pd.read_sql_query against a PostgreSQL or MySQL connection (sqlalchemy.create_engine), chunked reading with chunksize for tables too large to fit in memory, in-memory transformation, .to_parquet for the output (10x smaller than CSV, preserves dtypes including categoricals and datetime). Tutors include the dtype mapping that prevents int64 silently becoming float64 when NaN is present.
Series.str accessor for vectorized string operations (lower, strip, replace, contains, split, extract with regex), pd.api.types.is_string_dtype guards, conversion to the new PyArrow-backed string dtype for 2x speed-up, and apply with a custom function for operations not in the .str API. Tutors include the .str.cat versus + comparison.
Debugging
# chained selection, may not mutate df
df[df["price"] > 100]["tier"] = "premium"
# SettingWithCopyWarning firesdf.loc[df["price"] > 100, "tier"] = "premium"
# mutates df regardless of view/copy statusdf["squared"] = df["x"].apply(lambda v: v ** 2)
# ~2.4 s on 1M rowsdf["squared"] = df["x"] ** 2
# ~3 ms on 1M rowsInner merge on a non-unique key produces a cross product within the duplicates: 3 customer rows match 4 order rows on customer_id produces 12 combined rows. Use validate="one_to_one" or "one_to_many" or "many_to_one" on the merge call to assert the expected cardinality. Pandas raises MergeError if the assertion fails. Tutors check expected row counts on every merge in the deliverable.
Operations on a tz-aware column with a naive datetime literal raise TypeError. Either tz-localize the naive datetime (pd.Timestamp("2024-01-01").tz_localize("UTC")), or strip the tz from the column (df["t"].dt.tz_localize(None)). For comparison, .between with tz-aware bounds requires the column to be tz-aware in the same zone.
By default groupby drops rows where the group-by column is NaN. Pass dropna=False to keep NaN as its own group. Common pitfall on demographic data where missing region or gender values should be reported. Tutors verify by .groupby().size().sum() versus len(df).
pd.read_csv loads the entire file into memory with permissive dtypes (object for any column with mixed values, float64 for any int column with NaN). Fix: pass dtype={"id": "int32", "price": "float32"} to specify types, usecols=["id", "price"] to drop unused columns, parse_dates=["timestamp"] to convert in-place, chunksize=100000 to iterate over chunks, and .to_parquet for the output (PyArrow-backed, smaller, faster).
df["x"].apply(lambda v: v ** 2) iterates Python over every row, taking seconds for large DataFrames. Replace with the vectorized form: df["x"] ** 2 (microseconds). Other vectorized alternatives: .str.* for string ops, np.where for conditional, .map for value-to-value lookup with a dict, .replace for substitution, and .cut or .qcut for binning. Tutors profile with %timeit to confirm the speed-up.
df1 + df2 aligns on both index and columns, producing NaN where either side is missing. Resetting indexes (df1.reset_index(drop=True) + df2.reset_index(drop=True)) or using .values to drop the index for the operation, then wrapping the result back in a DataFrame, avoids alignment. Symptom: arithmetic between two DataFrames of the same shape returns mostly NaN.
Code Examples
import pandas as pd
orders = (
pd.read_csv("orders.csv", parse_dates=["created"])
.assign(total=lambda df: df["price"] * df["qty"])
.query("status == 'paid'")
)
# .loc with both selectors is the SettingWithCopyWarning-free pattern
orders.loc[orders["total"] > 1_000, "tier"] = "vip"
top = (
orders.groupby("user_id", dropna=False)
.agg(total_spend=("total", "sum"), order_count=("total", "size"))
.nlargest(10, "total_spend")
)daily = (
df.set_index("timestamp")
.resample("D")["value"]
.mean()
.ffill()
)
daily_7d = daily.rolling(window=7, min_periods=1).mean()Related
DATA 100 teaches the principles and techniques of data science across 14 weeks under Joseph Gonzalez, Narges Norouzi, and Lisa Yan (recent semesters), with co-development between EECS and the Division of Computing, Data Science, and Society. The course covers 6 modules: (1) Pandas and data wrangling, (2) exploratory data analysis (EDA) and visualization, (3) sampling and experimentation, (4) modeling and least-squares regression, (5) gradient descent and feature engineering, (6) classification with logistic regression, decision trees, and cross-validation. Languages and libraries: Python 3.11 with Pandas 2.x, NumPy 1.26, scikit-learn 1.4, matplotlib, seaborn, plotly, and statsmodels in select labs. The course assesses through 12 weekly Jupyter notebook homework assignments graded by Otter-Grader (an open-source autograder originally written for DATA 8 and extended for DATA 100), 2 projects (Project A1 housing-price regression, Project A2 spam-classification with logistic regression), a midterm at week 8, and a final at week 14. Lectures Monday and Wednesday at 5 PM in Wheeler Hall (or Pimentel for larger semesters), discussion Friday at varied times. Grading: 30 percent homework, 30 percent projects (15-15), 15 percent midterm, 20 percent final, 5 percent discussion attendance. The course is the second course in the Data Science major after DATA 8 (Foundations of Data Science) and is a prerequisite for DATA 101 (Data Engineering) and CS 189 (Machine Learning).
8 recurring assignments covered
Get help with DATA 100FAQ
Submit your pandas assignment and get a working, commented solution within 12 hours from a verified CS graduate. Plagiarism-free, line-by-line annotated, with a reproducible test suite where the rubric allows it.
Submit pandas Assignment