At a moment when the vol­ume of new infor­ma­tion about AI can be over­whelm­ing, Sig­nal and Cipher’s CEO & Chief Futur­ist Ian Beacraft believes it’s impor­tant to find and main­tain a sense of perspective. 

Speak­ing at the Next Gen­er­a­tion SAP Enter­prise Archi­tect Learn­ing Forum in Feb­ru­ary, Beacraft used the intro­duc­tion of the steam engine dur­ing the Indus­tri­al Rev­o­lu­tion as an anal­o­gy for our present moment, putting the audi­ence in the posi­tion of a con­struc­tion work­er con­fronting a then-new tool for the first time and fig­ur­ing out how to use it to gain pro­fes­sion­al advan­tage. ​“A crane or steam shov­el doesn’t just move earth faster” on its own, he explained. ​“You have to actu­al­ly redesign all the things that hap­pen around it for it to be effec­tive,” such as redesign­ing job sites and real­lo­cat­ing crew assignments. 

The moment we occu­py now, Beacraft said, is ​“just like the Indus­tri­al Rev­o­lu­tion, where our val­ue went from our phys­i­cal labor to our men­tal labor.” So, it’s not sur­pris­ing that sim­i­lar grow­ing pains are being expe­ri­enced by employ­ees nav­i­gat­ing the intro­duc­tion of AI at their work­places; it’s dur­ing this ear­ly peri­od that ini­tial increas­es in effi­cien­cy can often be decep­tive. ​“I might be able to work faster or at high­er vol­ume in cer­tain areas, but just because I’m doing that doesn’t mean I’m get­ting that net gain,” Beacraft observed. Instead, poten­tial­ly, ​“I’m push­ing fric­tion some­where else in the system.” 

How can employ­ees actu­al­ly real­ize the val­ue promised by AI? Beacraft sug­gest­ed five strategies:

1. Con­tin­u­ous­ly redesign­ing AI work­flow, restor­ing work­er agency
2. Lever­ag­ing human insti­tu­tion­al knowledge
3. Shift­ing from declar­a­tive intent prompts to imper­a­tive prompts
4. Using archi­tect­ed infor­ma­tion for orga­ni­za­tion­al sys­tems (AIOS) to cre­ate a gov­er­nance layer
5. Craft­ing new, more mean­ing­ful pro­duc­tiv­i­ty metrics

Design­ing the Work

How best to re-engi­neer work­flows affect­ed by AI imple­men­ta­tion? The answer can’t just be one way: Work­ers don’t just adjust to the AI, but then them­selves adjust AI to suit their work­flows as its oper­a­tors, engi­neers or archi­tects. Beacraft stressed the dis­tinc­tion between the lat­ter two roles: ​“Engi­neer­ing is, ​‘Can I design these work­flows from first prin­ci­ples?’ Archi­tect­ing is, ​‘Are we opti­miz­ing — for the right met­rics, the right KPIs, the right form of val­ue — in the first place?’” 

Because of the neces­si­ty of that work to cal­i­brate and opti­mize AI for work, Beacraft argued that gen­er­al­ized anx­i­ety about job loss due to AI imple­men­ta­tion is mis­placed: ​“AI will be most­ly focus­ing on doing the work and, to some extent, engi­neer­ing work­flows. This gives us the abil­i­ty to look at it from a much high­er angle.” 

He com­pared the adjust­ment process nec­es­sary to the prac­tice required to become a suc­cess­ful musi­cian: ​“It takes hun­dreds of reps to ingrain it in you, but if you spend enough delib­er­ate prac­tice on it, you’re unstop­pable.” Once work­ers adjust to agen­tic assis­tance, peo­ple can expand their work to focus on redesign­ing how their work actu­al­ly works. ​“We are no longer respon­si­ble for stay­ing in those silos. We indi­vid­u­al­ly have to be think­ing about how the work needs to be redesigned at all times. Every sin­gle one of us is in R&D. There’s no one who’s left out of that remit.”

Human Insti­tu­tion­al Knowledge

Before estab­lish­ing agen­tic work­flows, lever­ag­ing the human instincts and dis­ci­plines present with­in one’s work­force is essen­tial. ​“The most valu­able intel­li­gence you have in your orga­ni­za­tion is not in your data sets, it’s not in your stan­dard oper­at­ing pro­ce­dures (SOPs),” Beacraft said. ​“It’s inside the heads of the peo­ple that live inside your orga­ni­za­tion. It’s the habits, the rules of thumb, the guardrails, the notes, Slack threads — all this data across your orga­ni­za­tion that basi­cal­ly tells your orga­ni­za­tion how it runs.” 

Focus­ing on the com­par­a­tive mer­its of dif­fer­ent LLMs is miss­ing the larg­er pic­ture, he said: ​“The real pow­er comes when you start to dis­sect the X fac­tor of your com­pa­ny. How do you artic­u­late the way your teams work that is dif­fer­ent than the way the stan­dard oper­at­ing pro­ce­dures might actu­al­ly indi­cate?” These gran­u­lar, real-world learn­ings can cre­ate data sets that AI agents work with more effectively.

Declar­a­tive Intent Prompts Ver­sus Imper­a­tive Prompts

Trans­fer­ring that insti­tu­tion­al knowl­edge is only the begin­ning. To begin think­ing like an archi­tect, Beacraft sug­gest­ed start­ing with a sim­ple ques­tion: What kind of bot­tle­necks in work­flow hand­offs still exist sole­ly due to tech­no­log­i­cal con­straints that no longer apply? Specif­i­cal­ly, ​“If AI could do 80% of the work that we do today, what would we be design­ing for our humans to do instead?” Then, once those ques­tions gen­er­ate new answers, how can work­places adjust to the rapid changes that result? 

As Beacraft not­ed, with tools that improve every six months, AI’s impact on work­places is accel­er­at­ing: ​“How do we make sure that our roles, our gov­er­nance, our deci­sion-mak­ing, are keep­ing pace?”

One solu­tion involves shift­ing from a declar­a­tive intent prompt mod­el to an imper­a­tive prompt mod­el. In the cur­rent land­scape, users often offer detailed declar­a­tive prompts to AI, con­strain­ing its effec­tive­ness by treat­ing it as a junior devel­op­er with lim­it­ed oper­at­ing approach­es. ​“Often­times it would get pret­ty far, but then some­thing would break,” Beacraft observed. ​“It would have to come to me, and I would be respon­si­ble for debug­ging. I would be respon­si­ble for course-correcting.” 

As a counter-exam­ple of a pro­duc­tive imper­a­tive prompt, Beacraft offered the fol­low­ing: ​“Cre­ate an inte­gra­tion design that maps sales order fields from Sales­force to SAP S/4HANA. Define the mid­dle­ware rout­ing log­ic and write error han­dling sequence.” What that prompt trans­lates to for the AI agent is, after the prompt is com­plet­ed, ​“Check your work. Now, gen­er­ate test pay­loads for each edge case and val­i­date against SAP S/4HANA post­ing log­ic until all [the pay­loads] pass. Do not come back to me until you run this over and over and all of those cri­te­ria have been suc­cess­ful­ly matched.” 

In this process, ​“I have now tak­en a machine that nev­er gets tired, does not give up until suc­cess has been met, and I’ve giv­en it con­di­tions that aren’t so restric­tive. It still knows what suc­cess looks like, how to avoid fail­ure, and it can use all the tools at its dis­pos­al to get there.”

Cre­at­ing an AI Constitution

Anoth­er key com­po­nent of suc­cess­ful AI imple­men­ta­tion is cre­at­ing a gov­er­nance lay­er known as ​“archi­tect­ed infor­ma­tion for orga­ni­za­tion­al sys­tems,” or AIOS. In Beacraft’s anal­o­gy, the AIOS is the con­sti­tu­tion for your AI, and your agents are the cab­i­net. This defines the bound­aries while agents get things done. A con­sti­tu­tion is some­thing that changes slow­ly. It informs what the prin­ci­ples are of what you can and can not do, but it doesn’t inform exact­ly what you have to say. 

As an exam­ple of an AIOS in prac­tice, Beacraft offered a recent case study from GitHub, where one bot dis­cov­ered that anoth­er one had acci­den­tal­ly dis­closed a secret key. The error was imme­di­ate­ly caught and addressed, then a new pol­i­cy was pro­posed and inte­grat­ed with­in sec­onds. ​“Nobody in our orga­ni­za­tion told it to do that,” Beacraft not­ed. ​“It was not pre-pro­grammed into the sys­tem. There was no par­tic­u­lar source log­ic for this process at all. The sys­tems just knew to do this based on the fact that the gov­er­nance lay­er had already pro­vid­ed all the log­ic for how to under­stand these types of sce­nar­ios. Some­thing like this allows a very small team to work incred­i­bly prolifically.” 

As anoth­er case study, Beacraft cit­ed a pro­to­type built at a meet­ing over an after­noon with a group of region­al bank­ing exec­u­tives, none of whom had any AI prompt train­ing pri­or to the ses­sion. The team then launched into a design ses­sion that researched a finan­cial ser­vices pro­to­type, build­ing it along­side a brand and com­mu­ni­ca­tions plan, includ­ing a whole year of pod­cast scripts, the brand design guide­book, and a pro­to­type. A process that once might have tak­en six weeks could now be test­ed in an afternoon. 

“We’re now in a world where the pro­to­type costs less than the meet­ing to think about it,” Beacraft observed. ​“One per­son with three or four hours bang­ing on a pro­to­type can come to the table and not just say, ​‘Hey, I have an idea,’ but, ​‘Here is the expe­ri­ence. Here it is in prac­tice and appli­ca­tion.’ At that point, your ideas come in con­tact with real­i­ty, and you’re deal­ing with data, not assumptions.”

Mean­ing­ful Metrics

Cre­at­ing met­rics that mean­ing­ful­ly mea­sure AI’s input requires think­ing beyond met­rics that mea­sure the effi­cien­cy and scale of well-defined sta­ble sys­tems. ​“Every time some­one has an AI imple­men­ta­tion, they’re always focused on, ​‘Where’s the ROI in five min­utes? Why aren’t we 10 times faster right now?’” Beacraft noted. 

More mean­ing­ful met­rics might look like con­sid­er­a­tions of pro­to­typ­ing rates: ​“How many pro­to­types are com­ing from an indi­vid­ual or an orga­ni­za­tion or a team? What’s your pro­to­type-to-com­mer­cial­iza­tion ratio? How many of those are actu­al­ly being deployed? These are met­rics that are lead­ing indi­ca­tors of progress down the road. Ulti­mate­ly, if you build the sys­tems, you show what’s val­ued, you show what behav­iors are going to be reward­ed, so that we can all move for­ward into this new ter­rain and col­lec­tive­ly build that future.”

For more cov­er­age of the Next Gen­er­a­tion SAP Enter­prise Archi­tect Sum­mit, sub­scribe to ASUG­’s First Five newslet­ter and stay tuned for more cus­tomer sto­ries, expert inter­views, guest per­spec­tives, and ses­sion recaps in the weeks ahead.