Journey into Utility AI for Unreal Engine (Part Two)

Since the last time I wrote about Utility AI for Unreal Engine 4, it has been stream-lined and so has the Action System it’s built on. The Action System is quite similar to Unreal’s Gameplay Ability System for those familiar. Towards the end I’ll tease a few other AI related concepts I’ve been working on. Here is part one in case you missed it.

This post is mainly going to be a walkthrough of what I’ve been experimenting with and how the overall structure is shaping up. I’d like to add more practical guides in future parts including some more gameplay footage of the AI in action.

System Breakdown

Let me try to break it down: The AIController has an ActionComponent filled with the utility actions. These actions contain a scoring function and an execute function. The execution function is often basic as they often just trigger abilities on the Pawn they control.

This controller contains a custom blackboard too, allowing run-time adding/modifying of keys and values. This in particular is a little more powerful than the built-in blackboard which doesn’t let you add runtime keys unfortunately. Replication and SaveGame support are other neat bonuses. You can store the current target or any other runtime data for AI to make decisions and perform actions.

While scoring actions, you may need to keep track of which location or target was ‘best’. You simply score this inside the action itself as a regular ol’ variable. This way no other action interferes with this until you are ready to execute and set the new target in the controller’s blackboard.

Multiple tasks can now run at the same time, using resource locking to prevent incompatible tasks from running at the same time.

Resource Locking

A pretty interesting feature from AITasks in Unreal’s GAS is the concept of claiming resources. Such as claiming the legs of a character. So that you don’t attempt to run another task that requires the legs/movement. With the action system you assign which GameplayTags to apply to the owner on activation.

Example: A melee attack taking several seconds and locking the movement of the character. The action claims the resource Movement. A Dodge-roll task can no longer execute, as its set to require Movement to NOT exist on the AIController.

Runtime Blackboard

While Unreal comes with a Blackboard feature built-in for its Behavior Trees it lacks a few things I really want such as adding keys at runtime. Some other things I wanted: GameplayTags as keys (avoid typo’s, forgetting key names) Save Game support and possibly replication support. Replication is only relevant if Blackboards are used for things besides AI which I am still experimenting with.

The AIController contains a Blackboard and can be used as a data bank. This easily lets tasks share these variables such as TargetActor which might only be set by one task but used by many. Each key can be listened to for value changes.

Blackboards can be used for more than just AI. In games like Firewatch blackboards are used to store global story/game state that can be queried by quests, dialog etc. They spoke about their dialog system during GDC 2017.

Utility Query System

Some time ago I made a small EQS variation I dubbed UtilityQuery. It doesn’t have the fancy editor of EQS, but it’s simpler, and a lot easier to extend. A system like this (and EQS) is fantastic for spatial queries such as finding a spawn location for players, enemies, or treasure.

The query has a gather-step followed by a scoring-step similar to Utility-systems (and EQS for that matter). Again, this deserves its own post some time in the future. Here is what that could look like if applied as Perception replacement.

Bonus: Auto-matching Task name with Scoring Function.

For a while I used a C++ function that was able to run any Blueprint function by FName. This made it easy to setup tasks in Blueprint by name and match it to a scorer function in AIController. For example, my AI task LaunchMissile would match to the Blueprint function named Score_LaunchMissile automatically.

Can call Blueprint functions like Score_MyTaskName and return the function return value (BP function must return one float)

I used the following code. (disclaimer: I can’t guarantee the memory allocations used here are optimal or 100% safe – it’s the best I could find when looking around)

float ALZAIController::CallScoreFunctionByName(FName InFunctionName)
{
	UFunction* Func = FindFunction(InFunctionName);
	if (ensure(Func))
	{
		// Buffer is required to fetch the return value from
		uint8 *Buffer = (uint8*)FMemory_Alloca(Func->ParmsSize);
		FMemory::Memzero(Buffer, Func->ParmsSize);

		ProcessEvent(Func, Buffer);

		for (TFieldIterator<UProperty> PropIt(Func, EFieldIteratorFlags::ExcludeSuper); PropIt; ++PropIt)
		{
			// The "Score" return param was not marked as returnparm for unknown reason, the OutParm does work though so I am sticking with that one
			UProperty* Property = *PropIt;
			if (Property->HasAnyPropertyFlags(CPF_ReturnParm | CPF_OutParm))
			{
				uint8* outValueAddr = Property->ContainerPtrToValuePtr<uint8>(Buffer);
				float* pReturn = (float*)outValueAddr;
				return *pReturn;
			}
		}
	}

	UE_LOG(LogAI, Warning, TEXT("Failed to find and/or run scoring function by name '%s' for Actor '%s'. Check name and/or add return float parameter."), *InFunctionName.ToString(), *GetName());
	return 0;
}

Since then I moved over to embedding the scoring functions in the AI tasks themselves, making this feature redundant. I wanted to share it here regardless since I think it still has utility for others.

What’s Next?

AI is somewhat on the back-burner for my project as it’s working as intended. Providing decent gameplay challenges for the player as-is. I do eventually require 3D Pathfinding & Spatial Reasoning. I intend to completely replace all aspects of Unreal’s AI Module eventually (including navigation mesh).

As always, be sure to follow me on Twitter or check out my previous posts on AI.

References

11 Responses

    • It will come, once I evolve the system more so I have something to talk about. Meanwhile I’ve been using so it has already matured a bit since I wrote this last year.

      So far I certainly love it. Especially because the code for it has been so simple to maintain, and change as I discover what I need most. It also ties in super nicely with my own blackboard and utility queries.

  1. Hi Tom,

    Does this blackboard implementation for context mean that you lose out on the ability to change the task selection criteria via some fuzziness/randomization? For example, say you were selecting a random action from the top 5 – would this setup allow multiple actions of the same type? Like ShootAtTarget may have 3 different targets that all score within the top 5 possible decisions. Are you limited to a single optimal context for each decision?

    • The tasks themselves are still uobjects and can store variables like any other Blueprint. If that task needs to support multiple instances, you can store the ‘target’ in each of the tasks and let him use it. Then of course you can’t easily see every target for all the running tasks, but I don’t think this will be an issue (and running multiple tasks of a single class may already be unlikely – what examples are you thinking of)

      Any random fuzziness you can already handle in the task scoring too (which is what I do) you don’t need to store a top 5 and then only decide once the task decides to execute.

  2. Hi Tom Many thanks for the article.
    Trying to implement the system in my project and I am wondering how do you penalize an active action in order to give others a better chance.
    In other words, I see the system clearly when the AI should make an unique action at a time, but how about rotating within actions? How do you implement that?

    Example: At melee range -> attack -> block > try to strafe -> … –> back to attack
    All that sequence is an unique Action or each should be an unique action switched by some kind of logic?

    thanks!!

  3. Hi Tom!

    Really cool article. I’m currently working on my own implementation of an utility AI and recognized many of your thoughts, especially the one about resource locking. 🙂

  4. Hi Tom! thanks for this!
    I am using this kind of system in my project and evaluating its usefulness.
    I managed to use it with EQS and it is actually very useful and simpler than Behavior trees.

    However, the problem with which I find myself at the moment is that when two actions share almost the same requirements for scoring, the problems begin.

    Example:
    An enemy approaches the player, at a minimum of “X” distance, and then must flank him.
    When starting to flank, the distance to the player can grow a little and trigger the approaching action again. Creating a loop between these two actions and not completing any of the two.

    Of course I can solve it programmatically by creating flags, but I think there must be a more elegant and scalable way to do this.

    I think that the “Resource Locking” you mentioned could be an answer, but frankly I have no idea what it means or how to implement it.

    Did you find this problem in your way?

    Thanks!!

    • Hi Alexis,

      Resource locking is not intended for the problem you’re seeing. Resource locking is purely to avoid one task from running when another incompatible task is already active (eg. two tasks that want to move a pawn)

      You could look into Dave Mark’s video as he talks more about how scoring should work. You could try to use slight stickiness in the scoring, where a second task needs to score atleast 0.X higher to overtake the other task and get activated. So a task running with score 0.80 doesn’t get cancelled or overtaken by another one only slightly better like 0.82

Leave a comment on this post!