Solar powered dawn poems: progress report

Last year, CultureHub accepted a project proposal of mine for their residency program. I proposed the creation of “a radically small language model, trained on a low-power microcontroller,” which operates “only when drawing sufficient energy from solar panels.” This language model would be trained on a corpus of “albas, a poetry genre in which lovers lament the oncoming dawn” and generate new examples of poems in this genre—again, using only solar power. Both Solar Protocol and Low Tech Magazine’s solar powered server inspired this work, and I also see it as a continuation of my research in standalone poetry-generating devices.

In this post, I show the progress I’ve made on this project, and relate what I’ve learned so far. I also sketch a road map for future work to be completed as the residency draws to a close in a few months.

The big update is that I have successfully put together a prototype device that generates poems using only solar power. Here’s what my setup looks like right now on the breadboard:

I wrote a CircuitPython program on the Adalogger that reads from a corpus of poems on the Adalogger’s SD card, and generates new poems using a very simple Markov chain text generation algorithm. The Adalogger draws power from the supercapacitors by way of the AEMSUCA board, which “charges” the supercapacitors with energy from the solar panel. The generated poems are written back to the SD card as plain text files.

I tried to select components that provided only the bare minimum functionality for my goals, while still being common enough that other people could easily (and inexpensively) order the required components to reproduce my work. The total bill of materials comes out to under US$60, and many of the components can be swapped out with similar components that makers might already have sitting on their shelves. (Any 3.3v microcontroller could stand in for the Feather M0 Adalogger, for example.)

Power

One of my goals for this project was to have the device generate poems only when the sun is shining, drawing energy directly from the sun as directly as possible. But solar panels on their own don’t provide power consistent enough to reliably power a microcontroller. (Or, at least, I couldn’t find very many tutorials online about how to do this.) The typical way to deal with this problem is to use the solar panel to charge a battery, and then power the microcontroller from the battery. This would work fine for my purposes, but it’s more than I need: I’d end up with power to spare even when the sun isn’t shining. (Sure, I could have used a photoresistor or some other sensor to detect sunshine. But that would have taken the project even further from its original goal.)

What I needed, then, was power that comes from the solar panel, but “smoothed out” enough that I could use it to power a microcontroller without trouble. The AEMSUCA board was exactly the right solution for this. The board “harvests” energy from the solar panel into the supercapacitors, and when the supercapacitor’s charge is sufficient, provides an even 3.3v on its output pin. When the supercapacitors are tapped, the voltage stops. Even with one tiny 0.3 watt/2 volt solar panel, the AEMSUCA can charge the supercapacitors to capacity in a few hours, as long as the panel is in direct sunlight.

There’s something really magical about getting a computer (even a tiny computer) to do meaningful work with no wall wart, no USB cable, and no batteries—just the energy that you get from setting the breadboard out in the sun on the windowsill for a few hours. My setup isn’t really optimized for energy efficiency in any way, but I’ve found that I can run the microcontroller and generate poems for fifty to sixty seconds on a full capacitor charge. Not bad at all for a system that doesn’t have a battery!

Output

The original plan was for the device to transmit the generated poems over wifi. It turns out that wifi takes a tremendous amount of energy—more than the AEMSUCA could provide, at least—though it would be possible to use low energy Bluetooth or LoRa. While considering these options, it occurred to me that all of these methods for transmitting output had a characteristic that worked against my original goals for the project: they require another computer to be powered up, to receive the signal. But my goal is to make a standalone device that generates poetry with power from the sun, and if that device requires a second device to be active—a device that will most likely not be solar powered—then I haven’t really achieved my goal.

So I decided to use a simple SD card to store the generator’s output. This has the benefit of being entirely self-contained, and I like the idea of setting down the device in the sun somewhere, then coming back in a couple of weeks (or months, or years) to collect the poems it wrote.

There’s an unexpected drawback to this solution, however, which is that SD cards also use a lot of power. Even reading from an SD card can produce spikes of 100mA or more, which taxes the AEMSUCA and drains the capacitors quickly. This is especially troublesome, given the way that I’m generating the poems in my code (see below). I haven’t measured, so I don’t know for sure, but I’d guess that reading and writing to the SD card is using 80–90% of my power budget.

Programming

The Adalogger board supports CircuitPython, so I decided to use that for my prototype. I’m already very familiar with Python, and I was eager to get something working quick, so CircuitPython seemed like an ideal tool to use. And… wow, I really love CircuitPython! It’s powerful and easy to learn, and the workflow to program the device (just copy your .py file to a mounted drive) is incredibly convenient. Plus, being able to run a REPL on a microcontroller is a total dream.

I had fun playing with CircuitPython, and I was indeed able to make a working prototype of my text generation code very quickly. But there are some pretty significant drawbacks to working with CircuitPython, especially on the Adalogger board, which has only 256kb of flash memory and 32kb of RAM. First of all, the CircuitPython interpreter and associated libraries take up most of the flash memory. And once the Python interpreter is loaded, you really only have about 10kb of RAM to play with, which isn’t much. It isn’t enough, for example, to build a proper in-memory Markov model, even with a small corpus like mine.

So, I used a brute-force technique to generate text in a Markov chain-esque manner. (If you’re not familiar with how Markov chain text generation works, I have a little tutorial, or you can watch Dan Shiffman’s explanation. Dan is who first taught me about Markov chains!) First, I pick a random line of poetry from the corpus and grab the first n-gram from that line (I’m using n-grams of two words). Then I loop through every line of poetry in the corpus, trying to find another line containing that same n-gram. If I can find one, I tack the word following that n-gram on to the current output, and then loop through the entire corpus again, trying to find a line that contains the final n-gram of the current output. And I keep repeating that, until an entire line of poetry has been generated.

while True:
    # go to random start line
    curr = ""
    start_line_idx = randrange(line_count)
    line = get_line(start_line_idx).split()
    curr = " ".join(line[:n])
    print(curr, end=" ")
    while True:
        nexts = []
        for line in open(corpus_fn):
            try:
                idx = line.index(" ".join(curr.split()[-n:]))
            except ValueError:
                continue
            if len(line[idx:].split()) > n:
                nexts.append(line[idx:].split()[n])
        if len(nexts) == 0:
            break
        picked = choice(nexts)
        print(picked, end=" ")
        curr = "{} {}".format(curr, picked)
    print("\n")
    with open(output_fn, "a") as output_fh:
        output_fh.write(curr + "\n")
        output_fh.flush()

This works fine, but it is both excruciatingly slow and power-hungry (see above), since it has to read the entire corpus from the SD card for each new word. In practice, I can generate and save about three lines of poetry with this code before completely sapping the supercapacitor. That’s not enough lines of poetry!

So I think I’ll need to rewrite the whole thing in C. Hopefully, when I do this, I’ll be able to store my entire corpus in flash, making it much quicker (and less power-intensive) to access. Maybe I’ll even have enough RAM to actually make an in-memory model, which would also speed up the process considerably.

Corpus

I’m putting together a new corpus for this project, consisting of English-language dawn poems that are in the public domain (in the United States). Putting together this corpus has been the most labor-intensive part of the project, but also the most rewarding.

By “dawn poem,” I mean the genre of poetry which depicts lovers parting at dawn. This genre goes by many different names—alba, aubade, tagelied, among others—but is remarkably widespread across cultures and persistent across time. A well-known example in English literature is Donne’s “The Sun Rising” (on the topic of which please read Stephanie Burt’s lovely guide for the Poetry Foundation).

I’m interested in this genre for a handful of reasons. First, the genre is highly formulaic, and I want to discover to what extent (if at all) the semantics of the form can be captured using a simple language model, like a Markov chain. I’m also interested in seeing if a language model can produce variations on the form that reveal any latent characteristics, and produce poignant (or maybe just absurd?) new examples of the genre.

I also want to explore the tension between the subject of this genre (i.e., the sun) and the concept of solar power. There’s something a little bit tragic about material objects being called into service to create meaning, and I hope to evoke that tragedy by creating a device which is powered only by the sun, but which is also programmed to lament the sun’s arrival.

But above all, I’m drawn to dawn poems because they are, more often than not, simply beautiful. For example, consider this dawn poem, from Egypt in the 13th century BCE:

The swallow appears with the sun and threatens to separate this woman from her lover. But the woman dismisses the swallow, and decides to linger with her lover, despite the sun’s arrival. It’s a lovely poem and I get a warm fuzzy feeling whenever I read it. The poem and its translation appear in Arthur T. Hatto’s Eos, the only academic monograph on the topic of dawn poetry that I am aware of. Unfortunately, this poem is not included in my corpus, because it is not technically in the public domain. (Eos was published in 1965, well after the public domain cutoff in the US. I believe including the poem in this blog post is covered under fair use, however.)

Eos is a tremendous resource, and has hundreds of dawn poems drawn from many different languages and traditions. To construct my corpus, I’ve been going through Eos page by page and trying to find English translations for each poem that are in the public domain in the US. In some cases, this is easy: there is no shortage of 19th century translations of Ovid’s work (cited in Hatto, pp. 273–275). But for many of the cited poems, the only available English translations were made by present-day scholars, and are therefore under copyright—if there are English translations at all (aside from those provided in the text of Eos itself). I still have a ways to go in Eos, and then I have other bibliographies of dawn poems to explore. So far, I have forty-six poems, totaling 1540 lines and about 48,000 characters.

Finding public domain translations of these poems, or confirming that no such translation exists, takes a lot of time (and a lot of searches in the NYU Library web interface). But I’ve been enjoying the process of exploring these poems and poets. It has afforded me the opportunity to familiarize myself with texts that I may not have otherwise encountered. If not for this project, I would likely have never encountered the lovely, weird illustrations from the 14th century Codex Manesse:

Or the evocative Dramatis Personae of Peter Hausted’s “Rival Friends” (1632):

Steps forward

This project is a reaction to current trends in natural language processing research, which now veer toward both material extravagance and social indifference. My hope is that the project serves as a small brake on the wheels of these trends. I’m trying to demonstrate what it’s like to really think through the material and social preconditions for language models. What does it look like when you have to account for every watt of power you use, instead of feeding fossil fuels and dollar bills into the “cloud”? What does it look like when you have to account for every word in your corpus, instead of blithely chewing through terabytes of words that you stole from the Internet? What can be gained from this kind of care?

I hope that this project, even in its incomplete prototype state, helps to answer some of these questions.

But there’s much more to be done. Here’s a partial to-do list, both technical and conceptual:

Another task on the horizon is to make this corpus more, well, queer. As written, the poems afford straight readings nearly exclusively, and I’m not excited to propagate that worldview. I’m also not sure if the style of the poems in the corpus—rife with 19th century stilt and obsequious meter—suits my own aesthetics. I plan to edit (or even rewrite) the poems in the corpus to compensate for both of these factors, and to release both the original corpus and my rewrites under a creative commons/public domain license.

Wait, Allison. What about the poetry?

Oh right. So far, I have successfully generated three lines of poetry with this prototype. Here they are:

Not much, but it’s a start. Of course the generator had to work in a slightly lewd turn of phrase (in the second line), just to remind me that, yes, as always, I will have to read and edit the generator’s output before releasing it.