No. 44 — Prompt Engineering 🪄 Art Pedagogy 🎨 Evolution of Pokemon
Make a spell
My name is Linda. I write a bi-weekly (well, honestly more like monthly) newsletter about computer science, childhood and culture - and there are 9 716 of you listening. If you enjoy this issue, please share it with anyone you think may find it useful.
Sometime last summer I started hearing a new word: prompt engineering. It’s the science (and art!) of the kind of whispers, terms and keywords you give to an AI model to generate what you want - lately often a picture, heralded as art.
Writing a prompt has nothing to do with programming. It feels more like scrying or summoning. The joyful tricks that early adopters have discovered include things like simple repetition of the important thing: “homer simpson, from the simpsons, eating a donut, homer simpson, homer simpson, homer simpson”. Or including a pair of words like “in the style of David Hockney” or “in the style of Ghibli”. For some reason also adding “high quality” or “4K” will produce better results. So many images were generated - the #dalle tag on Twitter has an outburst of artwork.
I’ve had a hard time loving any of the end results.
It’s been hard to pinpoint why exactly - as a concept this computer/human creation is my world.
I think one of the reasons is that the works often lack the process and presence of the author that art education highlights. I think all artists have a deep curiosity about their materials. Are DALL-E, Midjourney, Stable Diffusion or other technologies an extension of Photoshop? Code rendered worlds? How should they be used in schools? Should the prompts and different versions be visible, like a brushstroke or provenance of a codebase? I don’t know yet.
Two years ago I wrote about the value of drawing as exploration (No. 08 — On Drawing 🖍️ What's Inside a Computer 📒 Sketching with a neural network) and more recently on collaborating with a computer perspective (No. 29 — David Hockney 👀 But to see like this 🐈 They All Saw a Cat). Last winter I outlined some thoughts on how code could whispers as an art material for kids. A “computational pen” could include at least the following:
Collage (“I made something new out of something old”). DALL-E, GPT-3, whatever the current new generative tool is. For art education the ability to build a Shakespeare-trained writing pen or a brush with the stroke style of Rembrandt is just so immersive.
Simulation (“I made a new imaginary world”). Unity/Unreal Engine, whatever the game engine of the moment is. Being able to simulate a world, a historic event, spread of a pandemic.. So much opportunity and an area where the computer is the perfect material.
Collaboration (“I’m working with other intelligencies across time/space”). Jupyter/Colab/Spreadsheets - or the likes. I think one of the most fascinating things about computers is the ability to collaborate with others - both physically distanced, but also across time.
Anyways, I think there is a lot of work for pedagogists to figure out how to introduce these tools to children. One of my favorite voices in this realm is the wonderful Art Pedagogy. I’m going to borrow a prompt list from one of their activities, but instead of coloring pens, paper cuts or acrylics, I’m hoping it will inspire also the engineers working on these technologies.
The visual prompt is simply the starting point, take this daily shape/doodle and do with it what you wish. You might:
- Develop or incorporate this into an abstract response (experiment, doodle ...) via drawing, painting, mini-sculpture, collage or textile-based response.
- Convert the shape into something else (recognisable, or not) by doodling or illustrating within or around it.
- Photograph something visually similar or connected.
- Research/discover/connect an artwork, concept or idea - and simply share this, or write about this.
- Use the shape as a prompt for creative writing, poetry, performance ...
In computer science, a linked list is a linear collection of data elements whose order is not given by their physical placement in memory. But here it is a selection of things I’ve been reading lately.
This was wonderful recap of a children's playlab with DALL-E - I love how the children were not impressed with the first round of images. It was only after the facilitators changed the medium to play-doh that the admiration arrived.
The only DALL-E project I’ve giggled over: this Randall Munroe thread on the evolution of Pokémon cards through history generates something delightful and original to the computer’s illustration/imagination abilities.
For more old fashioned spell-making supplies, I have three recommendations: Crayola Slick Stix, Ooly Chunkies paint sticks and Mungyo Gel color stix. All three have great pigment, colors that blend and are easy to clean. I only wish they came in more interesting color palettes! Holiday stocking stuffers for kids of very different ages.
I’m hoping to surface and share stories from all of you and I’d love to see your creations! Here are a few teachers using Ruby in creative, fun and inspiring ways.
The ever inspiring Irene and her Ruby toy ready for an adventure!
Artwork based on the second Hello Ruby book.
Pattern recognition activity for children to create their own fabric patterns. This could work well as a Holiday card decoration activity. From the first Hello Ruby book.