Final project | Notion

Objective

Complete a focused experimental project on a generative AI system, worth 40% of the course grade. Aim for about 20 hours of work, including experiments and writing.

Use an existing model, library, or small simulator. Established methods are welcome; model training is optional. Your contribution should be a clear question, a meaningful implementation or modification, and evidence that answers the question.

Possible topics

Topic	Example question
LLM question answering	Does retrieval improve answer accuracy? How does the amount of retrieved context affect the result?
Agent reliability	Does a verification or retry step improve task success, and at what cost in tool/model calls?
Multimodal generation	Does checking and revising a prompt improve object counts or spatial relations in generated images?

Other course-related topics, including generative modeling and small world-model experiments, are welcome in the proposal.

Required experiments

One task and one baseline: define the dataset, success criterion, and a simple reference method.
One proposed change: implement a focused modification, such as retrieval, a verification step, or a different sampling strategy.
One small ablation: remove a component or vary one setting to investigate why the result changes.
A fixed evaluation set: normally 30–50 held-out cases, shared across conditions. Propose an appropriate alternative for simulations or manual annotation. Keep other settings comparable and report runtime or model-call cost.
Error analysis: discuss at least five representative failures or boundary cases and the limits of your conclusions.

The baseline, modified method, and ablation are enough. Well-supported negative results receive full credit.

You may build on a homework, but add a new question or substantive method change and fresh evaluation cases. Identify what is reused.

Deliverables and dates

Proposal — April 15, 2027 11:59 PM (EDT): one paragraph (100–150 words) describing the question, baseline, proposed change, data, and available compute.
Final submission — May 13, 2027 12:00 PM (EDT): runnable code/notebook with brief instructions and saved results, plus a 4–6-page report in NeurIPS format, excluding references. Due at noon Eastern Time.

The report should cover the question, method, experiments, results, and limitations. Cite reused code/data and disclose AI assistance.

Grading

Problem and experimental design 20%; implementation 30%; evaluation and analysis 30%; report and reproducibility 20%.