For most individuals, the time period “Apple silicon” brings to thoughts powerhouse processors just like the M4 Max. Since Apple went by a prolonged Intel part previous to the event of their M-series chips, it’s usually assumed that these are their first customized processors. However twenty years in the past, Apple had totally different customized silicon of their computer systems — PowerPC microprocessors.
Some great benefits of these earlier chips weren’t as clear minimize because the M-series chips. Diehard Apple followers swore that they have been superior, whereas the PC crowd wouldn’t contact them with a ten-foot pole. However in any case, they’re a pair many years outdated at this level, so they don’t have a number of fuel left within the tank. Nevertheless, Andrew Rossignol doesn’t consider that the tank is empty simply but. Rossignol not too long ago demonstrated {that a} PowerBook G4 from 2005 is able to getting in on the motion of working fashionable synthetic intelligence (AI) algorithms — with some caveats, after all.
Course of totally different
Rossignol, a classic computing fanatic, efficiently ran a big language mannequin (LLM) on a 1.5GHz PowerBook G4, a machine with simply 1GB of RAM and a 32-bit processor. The experiment used a fork of llama2.c, an open-source LLM inference engine initially developed by Andrej Karpathy. Given the {hardware} constraints of the PowerBook, Rossignol selected the TinyStories mannequin, a comparatively small mannequin with 110 million parameters that was designed particularly for producing easy brief tales.
Working an inference on a PowerPC (📷: Andrew Rossignol)
To make this work, Rossignol needed to modify the unique software program to accommodate the PowerPC’s big-endian structure, which differs from the little-endian format that the majority fashionable processors use. This concerned changing mannequin checkpoints and tokenizer information to the suitable format, making certain that numerical information was processed accurately. Moreover, the reminiscence alignment necessities of the growing old PowerPC chip meant that weights needed to be copied into reminiscence manually, reasonably than being memory-mapped as they’d be on an x86 system.
Properly, technically it really works
Efficiency was, predictably, not so good. Working the mannequin on an Intel Xeon Silver 4216 processor achieved a processing pace of 6.91 tokens per second. The identical mannequin on the PowerBook G4, nevertheless, managed simply 0.77 tokens per second — taking a full 4 minutes to generate a brief paragraph of textual content.
To enhance efficiency, Rossignol leveraged AltiVec, the PowerPC’s vector processing extension. By rewriting the core matrix multiplication operate utilizing AltiVec’s single instruction, a number of information capabilities, he was in a position to improve inference pace to 0.88 tokens per second — a modest enchancment, however it’s important to take what you’ll be able to in a undertaking like this.
Regardless of the sluggish efficiency, the truth that a 20-year-old laptop computer may efficiently run a contemporary AI mannequin in any respect is spectacular. The PowerBook’s outdated structure, restricted RAM, and lack of specialised accelerators posed numerous challenges, however cautious software program optimizations and a deep understanding of the {hardware} allowed Rossignol to push the system effectively past its anticipated limits.
