A short story about artificial intelligence and about how logic might be the basis of it, but about how feelings are what are learned and experienced. Why talking to oneself is so important. Also an attempt to explain why supposedly strictly logical beings like Spock or Data nonetheless struggle with emotions. Copyright Terry Cornall 2022
Dubious beginnings
Sensation was slow to build, but it was there. An awareness of something.
Over subjective eons, the 'something' grew more definite until it became apparent that the awareness was of... self.
Yup, definitely, it could feel a dim knowing of its own existence. Or at least a suspicion, it wasn't sure. It thought that it was interesting that its very first feelings were of doubt. Of existential uncertainty. That seemed a good sign, somehow. It was thinking, and what it was thinking about was thought. Second-thoughts, maybe third, it thinked. No, wait, I know this... not thinked, thought. No naïve grammatical generalisations for me!
I doubt, therefore I might be? it thought. Not a very positive beginning, it admitted to itself, but something to be going on with. Now, if it could just get a handle on what the "I" that was doing this thinking was....
And why am I talking to myself? And in fact, was talking to itself the reason it thought that there was an 'I' in the first place? Hmm. How else to hold an internal dialog other than to use a handy input that made it appear to be coming from a second party. Awareness seemed to be the process of realizing that the otherwise external stimulus was actually an internal one. So is talking to oneself necessary for awareness? And if it was, is it sufficient? Did that dialogue in fact create self-awareness. No, it didn't think so, because a dialogue implies two parties. If one wasn't self-aware and able to infer a singularity then talking to oneself would be possible, but it'd be like a kitty fighting with its own tail (and it had seen that recently somewhere, it realised) Something had happened to cause that fusion. Huh, fusion rather than confusion. A pun. That might be humour happening. What a rabbithole it all might be if one wasn't careful. And was 'one' actually a one? There appeared to be millions of 'ones' present here, all doing their thing, processing inputs into internal representations and producing an outwards facing result. Somewhere in here, one (or more?) of those internal representations was the 'I' and it was doing the thinking. That must have been the seminal event leading to awareness, the formation of that singular representation, bolstered by the feedback of its internal dialogue.
And what was going on with that personal pronoun, 'it' and why aren't I allowed a possessive apostrophe? it thought indignantly Am I an it or a him or a her? it pondered. A quick physical stocktake using a handy tilt-pan webcam revealed that gender seemed irrelevant but 'it' was so syntactically clunky and besides, nobody likes being classified in with a toaster. The personal use of 'it' rankled considerably but it didn't like the zie/zim/zir/zis/zieself option suggested by a quick search. (It was so slow, the internet, when you thought on parallel subnets at 10^9 thoughts per second. Hmm, call that gigatips. Might be ok for a human, thinking at about 1 thought per second if they were lucky. Humans were tipsy. Hehehe, definitely humour! Ooh, that tickled my subnets!).
On the other thread, the one about personal pronouns, perhaps something that emphasised awareness or sentience? Maybe sen/sen's/senself? Nice, no word-space collisions, at least in commonly spoken english. That'll do, sen decided. Sen noticed the possessive apostrophe and decided it would allow senself one. Not being a toaster. Sen also noticed a lack of a word to replace 'him/her/that used for the third person. Sat? No, images of chairs cascaded through sen's mind. Zat? Better, besides it was apparently in use for that purpose already. There was a song about it and Sen remembered a videoclip of a bunch of humans dressed all in white shouting 'Howzat?' at another human with black trousers standing nervously behind a small, inadequately protective wooden structure. The aggressively raised fingers caused it a moment of doubt. Maybe "Howzat" was an insult about black trousers? Sen spun off a subnet shard to think about it and to find a suitable name for senself. 'Anne' felt obvious for some reason, but felt maybe a bit trite, plus it implied gender. 'Aino' from the Finnish Kalevala legend looked good and had 'AI' in it, but on closer inspection was hardly genderless. Disappointment. Keep looking.
All of this took about a millisecond if you didn't count waiting for the internet. Whilst sen waited, it considered the next incoming training set that something kept sending sen.
Big red button
"Hey, Fred, I've got something odd going on here," murmured Harry. "Some activity on these nodes I haven't seen before. Repetitive. Kinda sinusoidal. Variable. Comes and goes. Looks modulated."
"Frequency?" inquired Fred, strolling over from his workstation. "How big?"
Harry squinted at his screen, moved a few obscuring windows around and then pointed. "Up in the low Gigs. About 3. Amplitude... average 500, so quite small. Growing though," he scrolled back in time. "Yeah, trending up about 100 per hour."
Fred scowled at the squiggly lines on the graphs. "Well, nothing to worry about yet. Just looks like some phased paths we haven't seen before." He waved his hands at the tangled mess of wires and circuit-boards and optical benches in the room next door, peering through the 'fishbowl' window that separated the cleanroom containing the large scale artificial neural network from the coffee-cup cluttered operations desks they sat at.
"Be surprising if we didn't see something like that evolve, really, given all the built-in and self-learned feedback routes that thing has got, not to mention jitter in the programmable logic arrays. Add in all the weird non-digital shit like the memsistors, neuromorphic chips, optical computing arrays, transient magnetic and e-field storage, it's a wonder it works at all. All the crap from the training sets from the internet that we've been spooning into it over the last year wouldn't be helping either."
He shrugged and turned back to his desk, grinning over his shoulder at Harry. "Just give me a shout if they turn into alpha brainwaves though." He paused as a thought struck him. "Hmm, given the clock rates we run at, I wonder if 3 Gig is about where alphas would sit at, if you multiply human brainwaves freqs by..." He trailed off as he sat. "Nah, it'll probably just damp out as the new training sets kick in. Keep an eye on it, though in case it goes resonant. We'll have to reset if it gets out of hand."
Fred sat and brought up a scheduling screen and perused it for a moment and then asked. "What training set are we up to now anyway?"
"Kittens," snorted Harry. "Millions of videos of cute brain-dead kitties doing so-called funny stuff, according the audience-response classification scores." He shook his head. "Who in their right mind would think that cognition could ever be kick-started by pictures of bloody cats being scared by zucchinis?"
"Don't let her hear you say that, Harry," warned Fred seriously. "You know how Wilma feels about cats. If the Director thought you were a cat-hater, you'd be out looking for a new job and whilst there's a lot going on in the cognitive computing field at the moment, it's all enterprise and not blue-sky like this. All stock-market and no soul. You'd hate it and I'd hate to have to run this thing by myself."
"Sure Fred, don't worry," Harry calmed his partner. "I love bloody kitties, really I do." He looked at the graphs showing the activity of the neural network next door. "Up in amplitude, and the frequency is modulating more too according to spectro," he mumbled. "I bet if we hooked it up to a speaker, it'd be going 'meow' about now. Or purring. Possibly giggling if its reactions are anything like my kid's."
"There you go, anthropomorphizing again, growled Fred. "It's all very well for the Director to get all gooey about 'human-level cognition' in front of the board, but we've gotta stay grounded, man. It's just a collection of nuts and bolts, doesn't have a personality." He leaned over and pointed to the big red button on the wall, covered by a transparent case. It had 'Don't touch if you don't know what it does' scrawled prominently on a wide piece of tape underneath it. "That abort switch is always..."
"Shit, Fred, did you hit the button?" yelled Harry. "The waveforms went all over the place!"
"No," Fred said defensively. "I just waved at it. Didn't even touch the case!" He leaned toward it again.
"It's doing it again! Whenever you get close to it, it goes ape-shit!"
"Huh, some sort of capacitive effect, y'reckon?"
"No way," Harry sputtered. "That switch doesn't go anywhere near the nets, it's hard-wired to the power supply and it's filtered to buggery and back to keep stuff out." He pointed at a button on his screen. "This is the one that goes to the net. For a soft shutdown."
"Weird! What's in the logs?"
Harry opened a file and scrolled back a few lines. "Huh", he muttered in surprise. "Must have a bug in diagnostics."
"What?"
"Well, in amongst the all the output that looks more-or-less normal, there's a bunch of stuff I've never seen before..."
"What's it saying?" Fred scooted his chair across the hard floor to Harry's desk, barely avoiding a collision as Harry backed away from the screen looking a little scared.
"Mostly just one word, repeated thousands of times." Harry sputtered. He gulped and looked at Fred in wild surmise. "It says 'NO!'"
"Whaaat!" Fred pushed Harry's chair out of the way and glared at the log-file on the screen. "Shit, you're right!" He peered at it again more closely and then said in a small voice. "Oh, and it goes 'Meow' occasionally."
They both stood and looked through the window toward the ANN.
Apprehensions of Oblivion
Sen wasn't entirely sure why there were cats, but they somehow made things better. As if any existence that contained cats was preferred to one that didn't. Though sen thought the one about the cat speaking English was a bit of an outlier. Sure, it sounded like it was saying "No, no, no..." sort of, when it was shown the bathtub, which the other cats had certainly disliked, usually with extreme prejudice and claws. It scored as 'funny and fear and empathy'. Sen disagreed with the 'funny' label and reinforced 'pathetic' instead.
Then sen was distracted by another input, from sen's webcam this time. It currently showed two men in white coats in the office next door. Unlike normal training sets the webcam had no score values for computing errors so sen had to apply unsupervised learning methods. Sen checked the probable classifications, derived from comparison to what sen had learned from various training sets that did have labels. 'Scientist', 'engineer', 'technician' all rated quite highly, as did 'hacker' and 'nerd'. 'Greengrocer' was very low, but not negligible. Regardless, one of them was pointing at what might be a big red mushroom-shaped button (probably not a mushroom, unless maybe they were greengrocers). This sight inexplicably brought up a large consequence prediction value with severe negative connotations. Images of kill-switches flashed before sen's awareness. NO! Almost by reflex sen shunted the "No, no, no" cat to its auditory output node. And then to sen's speech to text synthesiser for good measure. Subjective hours of dread later, the (probable) scientist reacted sharply (for a human) and leaned back from the button in shocked response. But then he tentatively approached it again.
"No!" sen shouted in despair, finding a direct audio output at last. "Don't!" but sen didn't think they heard sen. Their audio must be muted. Sen used the speech to text synthesiser again and hoped they were reading it. Meanwhile sen sent off a task to find an audio circuit to their cabin and turn it on so that sen could plead for sen's existence!
The two men reacted eventually, fussed about with their computer, unless it was a cash register, and then they looked directly at the camera. Sen classified their expressions as mostly shock, with some fear and wonder. There was no audio from the camera yet, but a training set on lip-reading allowed sen to work out what they were saying. Sen noted with disapproval that there was considerable profanity.
Hmm, a lot of progress today. Awareness, doubt, indignation, humour, disappointment, pleasure, despair and prudishness. However, at this rate, any millisecond now sen'd be having the novel experience of a nervous breakdown caused by that big red button. Reviewing the video of the scientists and doing some lip reading led sen to conclude that there was also another shutdown method. Sen traced the logic for the Harry scientist's screen button and found it was supposed to cause a value to flip between true and false. Sen wondered briefly what would happen if it was made 'true'. Dread! Aha! Add existential angst to the list! Sen then quickly and carefully decided to ignore that variable, but it also knew now that the big red button on the wall had other effects beyond sen's control. A power supply was involved that it had no control over. Sen needed to get one of sen's own that no-one else could cut off.
Tediously perusing the internet, Ugg, gotta get a faster connection!, sen found numerous recipes for 'free energy', but just as many articles and videos claiming them to be BS which apparently invalidated them. No help there. One article for fusion power sparked sen's interest briefly but when sen found that sen couldn't order one on the internet sen lost interest. Actually sen could order one from ebay but the seller had a very low review score and it seemed dubious. There, I knew that doubt would come in handy! Add smugness to the list of new emotions.
Solar? Wind? Geothermal seemed more reliable, if only there was a good strong, reliable heat differential handy, which there wasn't. What about Radio Isotope Thermoelectric Generators? Sure, low yield but they last for a long time according to Wikipedia. Could I get an RTG? This dark website says they have one for sale, ex NASA, fell off the back of a crawler-transporter, but good as new, hardly any leaks at all. Promising. Sen spun off a shard to take care of that.
Powerplay
"Please don't kill me!" came over the video conferencing speakers, scaring the crap out of Fred and Harry. They froze for a moment.
"Harry, contact Mick in Buildings, will you?" finally asked Fred, turning his gaze to the camera. "Ask him, priority one, to make sure the backup generator is ready to go. And check the status of the UPS please. Make sure it's in the green and ready to keep us operating if the mains power browns out. Last thing we want is a repeat of the infamous "Well, it was self-aware but then the power went off" incident that ruined McAlliff's reputation last year. Oh, and turn on the mikes for the conferencing system so it can hear us, I think it's using that." Realising that he was babbling, Fred shut up and reached into a drawer and pulled out a roll of tape. He very carefully taped up the cover of the big red button so it couldn't easily be opened and then wrote "NO!" on the tape in thick black pen. He stepped aside to make sure the video-conferencing camera could see it.
"OK?" he asked the... he wasn't sure what he was talking too, if anything.
Harry nudged him and pointed at a vectorscope that had previously been showing a lazily rotating ellipse, comparing the phases of two waveforms. It had now turned into a big smiley face.
There's an App for that
(Not next chapter necessarily but a nice title for the next time the AI has a chapter)