Prompt Engineering

Prompt engineering is the process of designing prompts that elicit the desired output from an AI model. It is a general approach to using AI tools, and can be applied to a wide range of tasks. So far, my focus has been on prompt engineering of code, or at least GitHub repositories. Below is a schema I am developing.

What is Prompt Engineering?

Prompt engineering is about laying a digital trail of prompts that guide the AI toward what one wants to investigate, gradually refining. These are embedded in conversations one develops. It is possible, and now done regularly by experts, to develop a set of AI agents that respond to their own prompts and do part of their work. I don’t yet know how to manage such agents.

Prompts, and prompt engineering, is a useful way to develop tools without having to directly write code. A great guide is Tyson Swetnam’s Getting Started with Prompts. However, if you just use prompts and don’t keep track of them, you are missing an opportunity to easily document your work flow. That is, how will you later remember what you did and how? And how will you share your learning with others?

A simple way to begin is to ask your AI agent to save your prompts in a file, e.g. prompts.md. If you do more work within the same conversation, ask the AI agent to update the file. The next section shows how I used this approach to organize a somewhat complicated workflow. For some of my examples, see Sharing Prompts instead of Code and AI Prompt Examples.

Prompts to Organize Workflows

Workflows can get rather complicated. Often, it is helpful to organize them into human-size chunks to make them easier to manage and understand.

I experimented with developing a series of prompts to transform a workflow into its components. The functions do the work but need not be viewed in the workflow. The workflow should be a concise document that shows the steps and the results together. The following steps should begin with a workflow file, e.g. workflow.R. The workflow prompt is included in AI Prompt Examples.

Place the functions in their own R file (functions.R). This file should retain the comments that organize them into the workflow steps.
Organize the workflow into a Quarto document (workflow.qmd). This file should invoke (source) the functions.R and have its R code in chunks, with the comments turned into section delimiters and text blocks.
Modify the workflow to display results, with options to save images and tables as PDF and CSV files, respectively.
Present the workflow in source and rendered form to be readable by a human.

Later prompts would modify the workflow to have options to examine different datasets. Once I had understood these steps, I saw how I could have one comprehensive prompt that would orchestrate the process. I asked the AI agent to create this. These initial efforts are in the sysgenAnalysis repository, with the Workflow Prompt based on step-by-step instructions in the Workflow Walkthroughs

Learning about Prompts

I began learning about prompts from Tyson Swetnam, who demonstrated at the ESIIL Innovation Summit 2025 how a natural language paragraph could become an AI prompt. I adapted this with the Draft AI Task using Claude and MCP for the Three Rivers ESIIL working group, which became the section on AI Prompt Engineering. This exciting project is on hold but has great potential.