STATS 401 · Final project · CS201 homework process
The score has a past
Homework scores cluster near full marks. We followed online-judge submissions to see how differently students get there, and whether the path travels with the exam.
HW1HW2HW3HW4HW5
One turn per day, at the centre to ; clockwise from midnight.
1 dot = 1 submission in a visible window; grey arcs are withheld.
Why it matters
In of problem slots, every student who attempted the problem reached full marks in the end. What the 100 does not show is the first try, the retries, or the hour they arrived.
The online judge keeps that process in its submission log. Reading it helps students plan their own work, and helps TAs and instructors see where and when support is needed.
Questions
- When does homework work happen, and is a busy hour many students or a few students submitting again and again?Fig 1
- Do retries pay off, and does that depend on the homework, the time since the last try, or the point in the window?Fig 2 · 3
- What does a final 100 hide about how students got there?Fig 4
- Does the way students submit relate to their midterm score?Fig 5
Data & method
- SourceCS201 online judge, read-only snapshot of : course submissions from a -student roster; inside homework windows ( submitters, problem slots).
- CleaningTest account removed; each submission kept only inside its own homework window; scores scaled to 0–100 per problem; judge errors skipped.
- AnalysisFirst try, best within three, best observed; eligible retries; four-hour windows; five phases per homework. Exam: grades linked privately; Spearman ρ, 5,000-draw bootstrap.
- PrivacyOnly aggregates are published; any group of fewer than five students is withheld.
- Built withPython for cleaning; D3 v7, SVG and Canvas for the film and this poster. Online: switch views, sort, highlight a homework, follow a problem across figures, hover for counts, scrub the story.
Process how submissions and retries unfold
1When the work arrivesFilm · Figure 01
A burst can be many students, or a few students many times.
How to readOne bubble is one four-hour window, such as 29 Aug, 12:00–16:00, in which at least five students submitted. Further right: more students. Higher: more submissions per student. Bigger: more submissions in total.
What we seeThe largest visible window holds submissions from students; the most intense, from only , or each. Every published window falls between noon and midnight.
CarefulWindows with fewer than five students are withheld for privacy: submissions in windows, of them before noon. Counted, never placed, never zero. Clock time is not study time.
2Do retries pay off?Film · Figure 11
of eligible retries set a new personal best: %.
How to readAn eligible retry is a scored resubmission made while the student's best on that problem is still below 100. The dark bar is the share that beat every earlier score; on the right are the counts behind each share.
What we see% of these retries came less than a minute after the previous attempt, and they set a new best least often: %, against % after 1–10 minutes.
CarefulRetries are repeated events from the same students, not independent trials. Gaps are clock time, not study time, so these rates do not estimate an effect of waiting. Long gaps have few retries: read the counts.
3When in the window?Film · Figure 12
Retries progress unevenly from opening to deadline, and each homework has its own rhythm.
How to readEach row is a homework. Its open period, from release to deadline, is cut into five equal parts: 0–20% is the first fifth, 80–100% the last. A cell is the share of that part's eligible retries that set a new best (darker is higher), with the counts and S, the number of students.
What we seeVisible rates run from % to %, and no single pattern repeats across the homeworks.
CarefulHatched cells have fewer than five students: withheld, not zero. A phase lasts hours depending on the homework. Deadlines are the configured ones; individual extensions are unknown. No deadline effect is estimated.
Outcome score gaps and the midterm
4What the 100 hidesFilm · Figure 03
After three tries, of problems still trail their best score by more than 10 points.
How to readOne row per problem, averaged over the same students. Orange is what was still missing after three tries.
What we seeThe largest remaining gap is points, for (). The lowest first try, , was on ().
Careful"Best within three" is a cumulative best. It is not the score of the third submission, and not a causal gain.
5Does rushing show in the exam?Film · Figure 06
Last-minute rushing went with lower midterm scores.
How to readEach row is one correlation across the same students, each with one value for the behaviour and one midterm score; no one is split into groups. ρ (rho) compares the two rankings: −1 opposite order, 0 no relation, +1 same order. A bar that crosses 0 leaves the direction unclear.
What we seeRushing, the share of a student's submissions sent in the last 24 hours before each deadline, has ρ = (95% interval to ): the only bar entirely below 0. Leaving out any one student keeps it between and .
Careful"Rushing" names submission timing, not effort or study time. Grades were supplied for of eligible students: a partial, non-random sample, using pre-exam only. An association in a small group, not a cause and not a prediction.
What the 100 was hiding
- One score, many roads.Figures 1 · 4Full marks hide very different paths: busy windows and a few students submitting again and again; problems solved at once and problems still short after three tries.
- Retries progress unevenly.Figures 2 · 3% of eligible retries set a new best, at rates that differ by homework, by time since the last attempt and by phase of the window.
- Rushing went with lower scores.Figure 5Among supplied grades, more last-minute submitting went with lower midterm scores (ρ ): an association, not a cause.