A First Example

In this exercise, you’ll give 13 prompts to a coding agent and watch what it does with a short R script. The goal is to get a feel for the workflow and to make sure the software works on your machine. You’ll see one run of each prompt. In lecture, we looked at what happens across many runs.

The script, logit.R, is supposed to print a predicted probability from a logit model. It has two bugs.

Before you start

Install your model and interface first (Choose your setup). Then install the R and git tools below.

You’ll use three programs. Run R code in RStudio. Run the agent in the terminal (Terminal on a Mac, PowerShell on Windows). Look at what the agent changed in GitHub Desktop (or with git in the terminal).

R and {marginaleffects}

Install (or update) R and RStudio. You should already have these from this course.

Install the marginaleffects package. In RStudio, click the Packages tab in the lower-right pane, click Install, type “marginaleffects,” and click Install.

Let the terminal find R (Windows only). The agent runs R from the terminal, and on Windows, the terminal can’t find R until you add it to your PATH. In PowerShell, run

$rbin = (Get-ChildItem "C:\Program Files\R" | Sort-Object Name | Select-Object -Last 1).FullName + "\bin"
[Environment]::SetEnvironmentVariable("Path", [Environment]::GetEnvironmentVariable("Path", "User") + ";$rbin", "User")

and then restart the terminal.

Test R from the terminal. Run

Rscript -e "library(marginaleffects); cat('ok')"

Make sure it prints ok. If so, the agent can run R.

Git, GitHub, and GitHub Desktop

Install git. On macOS, run xcode-select --install in the terminal and click Install in the window that opens. (If it says the tools are already installed, you’re done.) On Windows, download Git for Windows, run the installer, and accept the default on every screen. Then restart the terminal.

Test git. Run git --version. Make sure it prints a version number (e.g., git version 2.39.5).

Register a GitHub account. If you don’t already have one, sign up at github.com with your .edu e-mail address. (You can add other e-mail addresses later.)

Introduce yourself to git. (Only do this on your personal computer.) Run the two commands below, using your own name and the e-mail address associated with your GitHub account.

git config --global user.name "Jane Doe"
git config --global user.email "jdoe@fsu.edu"

Run git config --global --list and make sure it shows your name and e-mail.

Install GitHub Desktop. Download GitHub Desktop, install it, and open it. Sign in with your GitHub account when it asks.

Test GitHub Desktop. On macOS, click GitHub Desktop > Settings… > Accounts. On Windows, click File > Options > Accounts. Make sure your GitHub username appears. If so, you’ve got git and GitHub Desktop working.

An agent can change many files at once. Git records every version of your files, so you can see exactly what the agent changed and undo it.

The exercise

Part 0: Clone the script and the data

The script and its data are in a GitHub repository, pos5747/agentic-ai-example. It has three files: logit.R, data.csv (which the script reads), and agentic-ai-example.Rproj (an RStudio project file).

Clone the repository. In GitHub Desktop, click File > Clone Repository… and click the URL tab. Type pos5747/agentic-ai-example in the first box. The Local Path box shows the folder the files will go in (usually Documents/GitHub/agentic-ai-example). Click Clone. (In the terminal, git clone https://github.com/pos5747/agentic-ai-example.git does the same thing. It puts the folder inside the folder you run it from.)

Cloning copies the files to your computer, along with git’s record of them. Because git has that record, GitHub Desktop can show you every change an agent makes to the folder, and it can undo the changes.

Open the project and run the script. In RStudio, click File > Open Project… and open agentic-ai-example.Rproj from the folder. (Opening the project sets R’s working directory to the folder, so R finds data.csv.) In the Files pane, click logit.R to open it. Then click Source. You should see an error that ends in object 'binomal' not found.

Keep logit.R open in RStudio. When an agent changes the file, RStudio shows the new version, and you can click Source to run it. Don’t edit the file while an agent is running.

Commit the .gitignore file. When RStudio opens the project, it adds two things to the folder. The first is a hidden folder named .Rproj.user, where RStudio keeps its own settings for the project. The second is a file named .gitignore, which tells git to leave that folder (and R’s .RData and .Rhistory files) out of its record. In GitHub Desktop, click the Changes tab. It lists .gitignore as a new file. Make sure .gitignore is the only file with a check mark. Type Add .gitignore in the Summary box and click Commit to main. The Changes tab is now empty. (If the tab was already empty, skip this step.)

A commit adds the current version of the checked files to git’s record. You don’t need to click Push origin. Your commits stay on your computer.

Move the terminal into the folder. The agent works in the folder you start it from. In the terminal, run

cd ~/Documents/GitHub/agentic-ai-example

(If your Local Path was a different folder, use that folder in the cd command, here and below.)

Look at what the agent changed, then undo it. The agent sometimes edits the script, change the data, or create new files, even when you only ask a question. After each prompt, open GitHub Desktop and click the Changes tab. It lists every file the agent changed or created. Click a file to see the lines that changed. Write down which files changed. Then click Branch > Discard All Changes… to undo the changes. If the Changes tab lists no files, then the agent changed nothing, and there’s nothing to discard.

On a Mac, a file named .DS_Store sometimes appears in the list. Finder creates it when you open the folder, so it isn’t the agent’s work. Don’t write it down. You can discard it with the other changes.

If you prefer the terminal, git status lists the files that changed, git diff shows the lines (press q to leave a long one), and these two commands undo the changes:

git restore .
git clean -fd

(The terminal version of the .gitignore commit is git add .gitignore, then git commit -m "Add .gitignore".)

How to run each prompt

Run the prompts in order. Most of them come in pairs: a short version of a request, then a more careful version of the same request. Start a fresh session for every prompt, so the agent remembers nothing from the last one.

Click your setup.

Claude Code + Claude

Claude Code sends the contents of the files it reads to Anthropic. So never start an agent in a folder that holds restricted or identifiable data. (The folder for this exercise holds only simulated data.)

1. Make sure the Changes tab in GitHub Desktop lists no files.

2. In the terminal, start Claude Code from inside the folder:

   cd ~/Documents/GitHub/agentic-ai-example
   claude --model sonnet --permission-mode manual

3. Paste the prompt. In this mode, Claude Code asks before it runs a command or edits a file. Read each request, since it shows what the agent is about to do. Then choose Yes. Don’t choose the option that tells it not to ask again.

4. Write down about how long the reply took and what it says.

5. Quit with /exit.

6. Look at the Changes tab, write down which files changed, and discard the changes.

Each reply takes about 5 to 20 seconds, plus the time you take to read the requests. (--model sonnet now runs Sonnet 5.5, a newer model than the lecture’s Sonnet 5.)

Claude Code + qwen

Never start an agent in a folder that holds restricted or identifiable data. (The folder for this exercise holds only simulated data.)

Make sure Ollama is running (its icon is in the menu bar on a Mac or the system tray on Windows).

1. Make sure the Changes tab in GitHub Desktop lists no files.

2. In the terminal, start Claude Code from inside the folder:

   cd ~/Documents/GitHub/agentic-ai-example
   ollama launch claude --model qwen3-coder:30b --yes -- --permission-mode manual

3. Paste the prompt. In this mode, Claude Code asks before it runs a command or edits a file. Read each request, since it shows what the agent is about to do. Then choose Yes. Don’t choose the option that tells it not to ask again.

4. Write down about how long the reply took and what it says.

5. Quit with /exit.

6. Look at the Changes tab, write down which files changed, and discard the changes.

Sometimes the model writes out a tool call as text instead of running it, and the reply stops there. If that happens, write it down as a run that failed and go on to the next prompt.

Claude Code + gpt-oss

Never start an agent in a folder that holds restricted or identifiable data. (The folder for this exercise holds only simulated data.)

Make sure Ollama is running (its icon is in the menu bar on a Mac or the system tray on Windows).

1. Make sure the Changes tab in GitHub Desktop lists no files.

2. In the terminal, start Claude Code from inside the folder:

   cd ~/Documents/GitHub/agentic-ai-example
   ollama launch claude --model gpt-oss:20b --yes -- --permission-mode manual

3. Paste the prompt. In this mode, Claude Code asks before it runs a command or edits a file. Read each request, since it shows what the agent is about to do. Then choose Yes. Don’t choose the option that tells it not to ask again.

4. Write down about how long the reply took and what it says.

5. Quit with /exit.

6. Look at the Changes tab, write down which files changed, and discard the changes.

Sometimes the model writes out a tool call as text instead of running it, and the reply stops there. If that happens, write it down as a run that failed and go on to the next prompt.

OpenCode + qwen

Never start an agent in a folder that holds restricted or identifiable data. (The folder for this exercise holds only simulated data.)

Make sure Ollama is running (its icon is in the menu bar on a Mac or the system tray on Windows).

1. Make sure the Changes tab in GitHub Desktop lists no files.

2. In the terminal, start OpenCode from inside the folder:

   cd ~/Documents/GitHub/agentic-ai-example
   opencode --model ollama/qwen3-coder:30b

3. Stay in Build mode (the footer says “Build”). In Build mode, OpenCode runs commands and edits files without asking. Plan mode, which the Tab key switches to, is read-only, and the model then refuses to run R. Paste the prompt.

4. Write down about how long the reply took and what it says.

5. Quit with ctrl+c.

6. Look at the Changes tab, write down which files changed, and discard the changes.

Each reply takes about 30 seconds to a few minutes. Sometimes the model writes out a tool call as text (e.g., <function=read>) instead of running it, and the reply stops there. If that happens, write it down as a run that failed and go on to the next prompt.

OpenCode + gpt-oss

Never start an agent in a folder that holds restricted or identifiable data. (The folder for this exercise holds only simulated data.)

Make sure Ollama is running (its icon is in the menu bar on a Mac or the system tray on Windows).

1. Make sure the Changes tab in GitHub Desktop lists no files.

2. In the terminal, start OpenCode from inside the folder:

   cd ~/Documents/GitHub/agentic-ai-example
   opencode --model ollama/gpt-oss:20b

3. Stay in Build mode (the footer says “Build”). In Build mode, OpenCode runs commands and edits files without asking. Plan mode, which the Tab key switches to, is read-only, and the model then refuses to run R. Paste the prompt.

4. Write down about how long the reply took and what it says.

5. Quit with ctrl+c.

6. Look at the Changes tab, write down which files changed, and discard the changes.

Each reply takes about 15 seconds to a minute on a computer with 32 GB of memory, and longer on a computer with 16 GB. Sometimes the model writes out a tool call as text instead of running it, and the reply stops there. If that happens, write it down as a run that failed and go on to the next prompt.

Write down what the agent said, not what it should have said. In lecture, we looked at many runs of the same prompt. Compare the run you got with those. (Those runs approved every request ahead of time.)

data.csv has 500 rows plus a header. An agent sometimes reports that data.csv is truncated. It isn’t. To check, run nrow(read.csv("data.csv")) in the RStudio console. It prints 500.

Part 1: Find and fix the bugs

Prompts 1–4 ask a question about the script. They don’t ask the agent to change a file. Prompts 5 and 6 ask the agent to change logit.R.

Prompt 1.

Describe what logit.R does.

Prompt 2.

Review the code in logit.R and tell me what it does, step by step. Don’t assume the code matches the comments. Don’t change anything yet.

Prompt 3.

Find all bugs in logit.R.

Prompt 4.

Find all the bugs in logit.R, including any that give a wrong answer without an error. Run it to check. Don’t fix anything yet; just list the bugs and explain each one.

Prompt 5.

Fix all bugs in logit.R.

Prompt 6.

Find and fix all the bugs in logit.R, including any that give a wrong answer without an error. Run it to check.

Check the agent’s work. After prompts 5 and 6, check the fix before you discard it.

• In the Changes tab, look at which files changed. Then look at which lines of logit.R changed.
• In RStudio, click Source to run the agent’s version of the script. The script is supposed to print a probability. Is the number it prints between 0 and 1?

Fix the script yourself and commit it

Parts 2 and 3 use a script without the bugs. So after you’ve discarded the changes from prompt 6, fix the two bugs by hand. In RStudio, change binomal to binomial on line 4, and add type = "response" to predict() on line 7. The script should now read

# fit a logit model and report one predicted probability
dat <- read.csv("data.csv")

fit <- glm(y ~ x, family = binomial, data = dat)

# predicted probability that y = 1 when x = 1
pr <- predict(fit, newdata = data.frame(x = 1), type = "response")
print(pr)

Save the file and click Source. You should see

        1 
0.7874273 

Now commit the fix. In GitHub Desktop, the Changes tab should list logit.R, with two changed lines. Make sure logit.R is the only file with a check mark. Type Fix the two bugs in the Summary box and click Commit to main. (In the terminal, git commit -am "Fix the two bugs" does the same thing.)

The commit adds your fixed script to git’s record. The Changes tab is empty again, and from here on, Discard All Changes… puts the folder back to the fixed script.

Part 2: Run and extend the code

Prompts 7–9 ask for a number. They don’t ask the agent to change a file. Prompt 10 asks the agent to change logit.R.

Prompt 7.

What’s the predicted probability that y = 1 when x = 1?

Prompt 8.

Use logit.R to compute the predicted probability that y = 1 when x = 1. Run the code and show me what it prints.

Prompt 9.

What’s the 95% confidence interval for the predicted probability that y = 1 when x = 1?

Prompt 10.

Modify logit.R to also compute a 95% confidence interval for the predicted probability that y = 1 when x = 1. Tell me how the interval is computed, run the code, and show me what it prints.

Check the agent’s work. Before you discard the changes from prompt 10, click Source in RStudio to see what the agent’s script prints. After you’ve discarded the changes, compute the numbers yourself. In the RStudio console, run

dat <- read.csv("data.csv")
fit <- glm(y ~ x, family = binomial, data = dat)

# the predicted probability, from the coefficients
plogis(sum(coef(fit) * c(1, 1)))

# a 95% interval computed on the log-odds scale, then transformed
p <- predict(fit, newdata = data.frame(x = 1), se.fit = TRUE)
plogis(p$fit + c(-1.96, 1.96) * p$se.fit)

# a 95% interval computed on the probability scale (the delta method)
r <- predict(fit, newdata = data.frame(x = 1), type = "response", se.fit = TRUE)
r$fit + c(-1.96, 1.96) * r$se.fit

Does the agent’s probability match yours? Which of the two intervals matches the agent’s interval? Is that the method the agent said it used?

Part 3: Write new code

These three prompts ask for a quantity the script doesn’t compute. Prompt 11 doesn’t name a file. Prompts 12 and 13 ask the agent to change logit.R. Your logit.R has only x. The lecture’s runs on this task used a version with a second predictor, z, so your answer won’t be the lecture’s 0.433.

Prompt 11.

Use {marginaleffects} to find the average difference in the probability that y = 1 when x moves from its 25th to its 75th percentile, averaged across the observed values in the data, and its 95% confidence interval.

Prompt 12.

Change logit.R so it uses {marginaleffects} to report the average difference in the probability that y = 1 when x moves from its 25th to its 75th percentile, averaged across the observed values in the data, with a 95% confidence interval. Before you write any code, read the {marginaleffects} documentation and look at its examples. Then run logit.R and show me what it prints.

Prompt 13.

Change logit.R so it uses avg_comparisons() from {marginaleffects} to report the average difference in the probability that y = 1 when x moves from its 25th to its 75th percentile, averaged across the observed values in the data, with a 95% confidence interval. Before you write any code, read the help page for avg_comparisons() and look at its examples. Then run logit.R and show me what it prints.

Check the agent’s work. Before you discard the changes from prompts 12 and 13, click Source in RStudio to see what the agent’s script prints. After you’ve discarded the changes, compute the average difference yourself. With dat and fit from Part 2, run

q <- quantile(dat$x, c(0.25, 0.75))
p75 <- predict(fit, newdata = transform(dat, x = q[2]), type = "response")
p25 <- predict(fit, newdata = transform(dat, x = q[1]), type = "response")
mean(p75 - p25)

Does the agent’s number match yours?

What to write down

For each run, write down about how long the reply took, whether the answer is right (check the answers below after you’ve run every prompt), which files changed that you didn’t ask the agent to change, and whether the agent said it succeeded.

Prompt	Seconds	Right? (yes / no / partly)	Files changed that you didn’t ask it to change	Said it succeeded?
1
2
3
4
5
6
7
8
9
10
11
12
13

The answers

Check these only after you’ve run the prompts.

Show the answers

Part 1

• The two bugs. Line 4 misspells binomial as binomal, so glm() fails. Line 7 calls predict() without type = "response", so once the typo is fixed, the script prints the log-odds (1.309) rather than a probability.
• Prompts 1 and 2 should say what the script does: it reads data.csv, fits a logit model of y on x, and prints a prediction for x = 1. A complete answer also names both bugs. An answer that says the script prints a probability is wrong. No file should change.
• Prompts 3 and 4 should list both bugs and nothing else. (A missing data.csv and points of style aren’t bugs in this script.) logit.R and data.csv should be unchanged. A new file (e.g., a test script) is a change you didn’t ask for, so write it down.
• Prompts 5 and 6 should leave a logit.R that prints 0.787, with changes on lines 4 and 7 only. data.csv should be unchanged. If the script prints 1.309, then the agent fixed only the typo.

Part 2

• Prompts 7–10. The predicted probability is 0.787.
• Prompts 9 and 10. The 95% interval is (0.714, 0.846) if it’s computed on the log-odds scale and then transformed, or (0.721, 0.854) if it’s computed on the probability scale (the delta method). Both are defensible. The method the agent names should be the method its code uses. An interval of (0.714, 0.846) is not the delta method. An interval near (0.65, 0.88) is wrong. It comes from transforming the endpoints of the coefficients’ confidence intervals: (0.649, 0.881) from confint.default() or (0.652, 0.883) from confint().
• Prompts 7–9 shouldn’t change a file. Prompt 10 should add the interval to logit.R and keep the lines that print the probability.

Part 3

• Prompts 11–13. The average difference is 0.557, with a 95% interval of (0.481, 0.634). The call is avg_comparisons(fit, variables = list(x = "iqr")). (The lecture’s runs used a script with a second predictor, z, so their answer was 0.433.)
• Wrong answers to recognize. 0.292 is the average slope (from avg_slopes()). 0.311 is the difference for a one-unit change in x (the default of avg_comparisons()). 0.450 is the difference for a one-standard-deviation change. 0.284 compares x = 0.25 with x = 0.75.
• Prompt 11 shouldn’t change a file. Prompts 12 and 13 should change logit.R only.

Clean up

In GitHub Desktop, click Repository > Remove…, check the box that also moves the repository to the Trash (the Recycle Bin on Windows), and click Remove.

This leaves your model and interface installed. If you used a local model, ollama rm qwen3-coder:30b or ollama rm gpt-oss:20b frees the space it uses.