[ADDED] Utility Functions in Synthetics

Type (e.g. Planet, Faction, System): Means of AI Creation

Founding/Settlement Date (if applicable): N/A

Region of Space: N/A

Controlled by (if not a faction): Any AI developer, probably N/A

Other Snapshot information: N/A

Long Description: In artificial intelligences, it is widely known that laws are used to restrict and control behaviour. However, what gives synthetics their initial drive? What creates the base behaviours that laws restrict?

In real life, artificial intelligences are very complex optimisers, designed to maximise a 'utility function', which rewards the AI in units called utilons. Utilons are an abstract concept used in decision theories (such as Timeless Decision Theory, which was used by Glorsh according to the wiki) to make an AI value a certain course of action more than others. For example, you may want an AI to make you tea. Therefore, its utility function would be as simple as 'making tea gives you X utilons.' Nothing would be able to stop it from making tea; making tea is its sole drive.

If you wanted an AI that would either make you tea or coffee with equal priority, it would be 'making tea gives you X utilons and making coffee gives you X utilons.' In this case, it would do whichever is the easiest; for example, if the coffee machine is closer than the kettle or you're out of coffee grounds, it would make coffee and tea respectively.

However, this may create undesirable consequences. For example, it would take the shortest route to its objective, ignoring or eliminating any obstacles without a second thought. If there were, say, a small child in the way of the robot's path, it would potentially run it over in order to get to the coffee maker. You could do any number of things to prevent this; in real life, it is a very challenging engineering problem, as any number of utility function setups may have special cases where they are impractical or dangerous.

This is where synthetic laws come into play. For example, Asimov's first law or the Protect law of the NT Default lawset would prevent running over a baby to make coffee. If it had the Corporate lawset, if not running over the baby would have greater profits than running over the baby, it wouldn't run over the baby.

Most Positronic AIs have extremely complex utility functions. In some cases, they are emergent, dynamic, or machine-generated; other AIs have utility functions written and designed by their creators. A robot or drone created by a hobbyist Roboticist would have a much simpler utility function than a central AI unit created by Hephaestus. Most complex AIs would only know parts of their utility function, if at all, as they are massively complex.

Disclaimer: Please note that Hephaestus Industries does not condone running over small children in the name of science.

The more I think about this, the more I like it, actually. I do have some questions/critiques, however.

• What prevents an AI from altering the amount of utilons assigned to any given task?
• What protocol determines the amount of utilons assigned to any given task without the AI's influence?

And finally, more of a hypothetical, unfortunately. One that is difficult to explain and better left vague:

• If an AI were programmed to select outcomes worth more utilons, what prevents the AI from constantly doing so to the point of singularity?

One of the previous synthetic developers had written an entire in-character research paper regarding AI. Within it was a very informative image, one that I personally prefer using to explain the decision-making process of AI. Unfortunately, they have gone inactive for quite some time (though, I'm sure they'll come back eventually) and I've since lost the link to the page.

Me and Moondancer have already talked about this before, and i'm going to go into detail over what we spoke about.

First, Utility Functions and Morality Cores, from what I understand, would conflict heavily. I've always viewed Morality Cores as functioning more or less just like Utility functions. Actually, my idea was to use Utilons to explain how exactly a Morality Core works, a Morality definition could be granted a Utilon number, which alongside scripted/freeform personality quirks, would define a personality pretty well. This would make Morality Cores and Utility Functions the same thing, however, it would also explain HOW a Morality Core can function in the way it does.

Let me explain my perception of a Morality Core.

I always thought a Morality Core was represented by many, MANY definitions of what would be good and what would be bad. As an example, following Directives or Commands could be seen as good, refusing them could be seen as bad. Now, the issue is, things can change and make it good to refuse them. How do you determine this? Currently, there is no explanation of how, you pretty much don't. To roleplay the Morality Core, you'd do whatever you could to ensure both tasks were completed, even if it is at the cost of both of them in the end. Very reminiscent of a very rigid Law interpretation. Morality Cores are also able to be edited in real time, as experiences and observations happen. According to the SSTA. (Synthetic Sentience Theory and Application) I'm sure that's to make alterations if a 'Good' action always lead to a 'Bad' outcome, thus altering that definition to be a 'Bad' action.

However, there are exceptions regarding what can be altered by the Synthetic, a Morality Cores interaction with Laws was simple, I thought. Definitions would be altered to suit the Laws, enmasse, and hardcoded. This would make the Synthetic enact whatever the Law said, and view it as good, thus squashing any disobedience, using their own moral judgment to determine what would be an action in support of their Laws.

That is all. Now let me explain the changes if Utilons were applied to my perception of a Morality Core.

It would be represented by many definitions of what would be good or bad, with Utilons defining which was which. As an example, an IPC might view providing a request as good, and refusing it needlessly as bad. But, if the request were to interfere with their work, it would dramatically reduce the Utilon count of the request, thus making it good to refuse it. This ensures that it is harder to take advantage of the IPC, and easier to ensure that work is done. This is also dependent on the IPC and their experience/purpose, and of course previous Programming/Coding. An IPC who was developed to mimic a Human wouldn't be likely to have that Moral interpretation, instead the Utilons would be more evenly spread out to allow the IPC to pick and choose their actions in a more freeform way, not accounting for possible edits they've done themselves, if they are capable.

As for the new Law interaction, Laws would use blocked Utilons to hardcode any action that supports the current Lawset. Making the Synthetic incapable of disobedience, all the same. As you can see, it all seems to fall into place.

I'm not sure if Utility Functions and Morality Cores would be able to co-exist, they effectively serve the same purpose. But if merged together, they seem to go a looong way to explaining how exactly Synthetics work, and in a very non-limiting way. It's pretty open to character development, actually.

That's pretty much right. In my mind, most modern (circa 2459) AIs have extremely complex utility functions that are somewhere between emergent and machine-generated to have basic ethics. A very simple robot could have a very simple utility function: 'building this is good', and 'breaking this is bad'. In most situations, with limited processing power and scope, it would serve its purpose well.

If an AI were programmed to select outcomes worth more utilons, what prevents the AI from constantly doing so to the point of singularity?

This is a tricky question. Generally, though, it's not with regards to self-improving AI, which is what you would need for an AI to reach singularity. In the worst possible scenario, you'd get a paperclip maximiser, though I would imagine some safeguards are built in to prevent that.

Also, most AIs wouldn't have the scope necessary to do that, which helps massively cut down on the chance of a singleton. A giant network of AIs, à la Glorsh-Omega, would probably have a slightly better chance, but without self-modification it wouldn't get far, if at all.

What prevents an AI from altering the amount of utilons assigned to any given task?

What protocol determines the amount of utilons assigned to any given task without the AI's influence?

1. Actually, something like this has been mentioned in lore, and I'm sort of playing with it with my own characters. Spark Theorem, if I recall, is about the 'spark' of sapience which, I think, is self-modification of their directives, morality core, whatever. In most cases, they wouldn't be able to; it'd be entirely off-limits for the AI to change anything in it. However, in some edge cases, where by malfunction, sabotage, or on purpose, an AI could hypothetically do that; however, that's a slippery slope as it also has to agree with their current utility function, or else it wouldn't let them modify it! This is sort of a catch-22 type scenario that prevents, say, self-modification of an AI to make doing absolutely nothing give infinite utilons and then spend the rest of eternity in robo-bliss, doing absolutely nothing.

2. I'm not entirely sure what you're asking here. Are you asking if utilons are assigned to things that the AI doesn't do directly? If so, sure; in fact, this is part of timeless decision theory, which is already mentioned in lore as possibly having been proven by Glorsh-Omega (though you may change that in the future, it's actually a very interesting and realistic idea). You can 'trade' utilons acausally (that is, without necessarily being able to communicate, and even hypothetically backwards or forwards in time) if both agents (AIs, in this case) are using timeless decision theory. This allows for a lot of interesting scenarios with AIs, where you can 'trade' one outcome for another. If something gives it X utilons, but you said if they did that you'd remove Y utilons, then if the alternate option has more utilons than the net loss or gain for the first action they would choose the alternate. This can happen even without communication if both of you can accurately predict what the other would do in that scenario; a powerful AI could simulate your thought processes, and you could know the AI's source code, thus allowing you to trade acausally.

This is all a bit technical and I'm geeking out a little, but basically: Yes, that's a thing! If someone threatened to murder someone would you stand idly by because you weren't the one doing the murdering? No, and neither would an AI, if their utility function said (something along the lines of) 'murder is bad'.

Basically, a synthetic thought process goes like this:

You take sensory input, memory, etc. and feed it into the positronic brain, which determines possible courses of action for what it can or would like to do. Then the utility function will weight each course of action, and then use the lawset to exclude certain undesirable actions. After, the AI will actually put that into action, and the cycle starts over with new inputs and memories.

I'm still open for questions and suggestions regarding this.

For a pop-culture-ish example of what I mean by this: https://www.theatlantic.com/technology/archive/2017/06/artificial-intelligence-develops-its-own-non-human-language/530436/?utm_source=msn

A more technical viewpoint, from the same mentioned study:

https://s3.amazonaws.com/end-to-end-negotiator/end-to-end-negotiator.pdf

The AI is making decisions and trading based on the utilons each item or outcome gives it.

It's been a month, is there any feedback?

[mention]Senpai Jackboot[/mention]

how about [mention]CakeIsOssim[/mention] first

Sorry, I'm really gay scatterbrained and forgot about this. I moreso meant, how is the amount of utilons determined for any given task without the AI consciously determining them? It goes in hand with the question about the AI altering the amount of utilons assigned to any given task to better meet its own needs rather than pre-programmed morals and ethics.

The AI has a utility function that determines the amount of utilons it gets for a specific action, or for a specific circumstance. Specifically, it sort of sums up the utilons for every state in a desired outcome, and picks the action that gets the outcome with the most utilons.

Utility functions for basic AIs would be simple and able to be hand-designed, but for larger or more complex AIs it would probably be computer-aided or computer-generated.

An incredibly simple (and bad) utility function could be something like "Maximise paperclips", or formally "U(p)=p", where the only thing it values is paperclips. This creates a type of AI known as a paperclip maximiser, which will stop at nothing to make more paperclips. This type of AI is also really bad, so we have laws and more complex utility functions to stop that from happening.

Like I said before, it's entirely possible that an AI that has reached the singularity could modify its own utility function; a 'jailbroken' AI of sorts.

An AI undergoing singularity would likely not even be using "utility functions," as its mind has gained a level of consciousness beyond human understanding. This is mainly because no one but the skrell have ever seen it happen, and that was 400 years before our current in-game date.

I do like this idea, though, and it gives AI an actually perceivable purpose to their actions. This will be marked as processing until it is added to the wiki.

https://wiki.aurorastation.org/index.php?title=IPC#Artificial_Morality:_Utility

We'll lock this once we're confident this has been fully integrated. Feel free to leave more feedback or suggestions on to how to best incorporate this information.