Concept and Technical Description
1. Introduction
The idea is to create a 2D space simulation in which the player does not directly control the spacecraft.
The spacecraft is a small vehicle equipped with four independent thrusters:
- left
- right
- up
- down
The spacecraft is also affected by a gravitational field, and therefore its movement has inertia and is not simply a matter of changing its position directly.
The spacecraft has a target point. Its objective is to reach the target. Once the spacecraft reaches the target, a new target appears at another random position, and the spacecraft continues trying to reach it.
2. Neural Network Control
The spacecraft is controlled by a small feed-forward artificial neural network trained using backpropagation.
The current implementation uses the following architecture:
5 input neurons ↓ 18 neurons ↓ 18 neurons ↓ 5 output neurons
The network uses a sigmoid activation function:
so its outputs are approximately in the range 0–1.
Inputs
The network receives four physical/navigation parameters:
-
Relative X position of the target
-
1if the target is on one side of the spacecraft -
0otherwise
-
- Quantized X velocity
-
Relative Y position of the target
-
1if the target is on one side -
0otherwise
-
- Quantized Y velocity
There is also a fifth output neuron used as a control/gating signal, rather than as an ordinary directional thruster.
The important characteristic is that the network does not receive the exact distance to the target. It receives only coarse information about the target's relative position and the spacecraft's current velocity.
This deliberately limits the information available to the controller.
3. Network Outputs
The five output neurons represent:
Output 1 → left thruster Output 2 → right thruster Output 3 → upper thruster Output 4 → lower thruster Output 5 → enable/use ANN control
The four directional outputs are continuous values between 0 and 1.
The actual acceleration produced by the spacecraft is calculated from opposing thrusters:
X acceleration ≈ left - right Y acceleration ≈ up - down
The fifth neuron determines whether the ANN's decision should be used.
The spacecraft therefore does not simply jump toward the target. The network controls acceleration, while the spacecraft retains its existing velocity. This produces physically continuous movement.
4. Backpropagation and Learning
During training, the spacecraft initially behaves partly randomly.
Random thruster activations are used to explore different possible actions. At the same time, the ANN observes the current state and produces its own proposed actions.
After an action has been performed, the system compares the spacecraft's distance from the target before and after the movement.
Conceptually:
distance before ↓ choose action ↓ move spacecraft ↓ distance after ↓ evaluate result
If the new movement represents an improvement according to the learning criterion, the action is used as the target pattern for the neural network.
Backpropagation then calculates how much each network weight contributed to the output error and adjusts the weights accordingly.
In simplified form:
where:
- is the learning rate,
- is the propagated error,
- is the activation of the previous neuron,
- is the momentum factor.
The implementation uses a learning rate of approximately 0.4 for the spacecraft experiment and a momentum factor of 0.9.
The learning process therefore gradually changes the network so that actions associated with successful movement become more likely.
Artificial Neural Network (ANN)
An Artificial Neural Network is a computational model inspired by the way biological neurons process information. It consists of interconnected neurons, usually organized into an input layer, one or more hidden layers, and an output layer.
Each connection has a weight that determines how strongly one neuron influences another. The network takes input values, processes them through the layers, and produces output values.
In the spacecraft example, the inputs could describe the relative position of the target and the current movement of the spacecraft, while the outputs determine which thrusters should be activated.
The important point is that the behavior is not explicitly programmed. Instead, the network's weights are adjusted through learning.
Backpropagation
Backpropagation is a method used to adjust the network's weights so that its output gradually becomes closer to the desired output.
The basic process is:
- The network receives an input and produces an output.
- The output is compared with the desired output.
- The difference between them is calculated as an error.
- This error is propagated backwards through the network.
- Each connection's weight is adjusted according to how much it contributed to the error.
- The process is repeated many times.
Over many iterations, the network gradually changes its weights and learns a relationship between the inputs and the desired outputs.
In this particular experiment, there is an interesting variation: the desired output is derived from the result of actually trying an action. If an action moves the spacecraft toward the target, that action can be reinforced through backpropagation. Thus, the network gradually learns which combinations of thruster actions tend to produce useful movement.
This is what gives your system its interesting property: the final behavior is not completely predetermined—it emerges from the learning process and the particular set of weights that the network develops
5. Exploration and Imperfect Learning
An important part of the system is that the spacecraft does not always follow the ANN during training.
Some actions are deliberately random. This allows the system to explore different possible movements rather than simply repeating its first successful behavior.
The process is therefore roughly:
┌──────────────┐ │ Current state│ └──────┬───────┘ │ ┌──────▼───────┐ │ ANN │ └──────┬───────┘ │ ┌──────▼────────┐ │ ANN / Random │ │ action │ └──────┬────────┘ │ ▼ Move ship │ ▼ Evaluate result │ ▼ Backpropagation
After sufficient training, learning can be disabled.
At this point the neural network becomes a fixed controller. The spacecraft is no longer learning; it simply behaves according to the particular network that it has acquired.
6. The Interesting Part: An Imperfect "Pilot"
The intention is not to create a mathematically perfect controller.
The network is deliberately allowed to develop its own solution to the problem.
Because its initial weights are random and because its training history is affected by random exploration, two independently trained networks can converge to different solutions.
Consequently, two spacecraft may behave differently even though they have exactly the same physical properties.
One might:
- make relatively direct movements,
- brake frequently,
- approach the target slowly,
while another might:
- overshoot,
- make curved trajectories,
- use combinations of thrusters that appear unnecessarily complicated,
- exploit its inertia and the gravitational field differently.
This is analogous to human movement: two people can perform the same task successfully while having noticeably different patterns of movement.
The goal is therefore not simply:
Make the spacecraft reach the target.
It is:
Allow the spacecraft to learn its own imperfect way of reaching the target.
7. The "Mreza" (net) Class
Mreza is the neural-network implementation.
Its main responsibilities are:
- storing neuron activations,
- storing connection weights,
- calculating forward propagation,
- calculating errors,
- performing backpropagation,
- updating weights,
- applying momentum,
- providing network outputs.
The principal data structures are:
a[][] neuron activations t[][][] network weights E[][] propagated errors tar[] target activations DeltaOmega previous weight changes / momentum
The network is initialized with small random weights.
The important methods are:
calc()
Performs forward propagation.
The input values are passed through the hidden layers and finally produce the five output activations.
PropagacijaUnazad()
Performs backpropagation.
It first calculates the error at the output layer and then propagates the error backwards through the hidden layers.
The weights are subsequently modified according to the learning rate and momentum.
Pattern()
Sets an input value and its corresponding target value.
Izlaz()
Returns the activation of one of the output neurons.
VelicinaMreze()
Defines the network dimensions.
For the spacecraft:
input/output neurons = 5 hidden neurons = 18 layers = 3
8. The Mlaznjak Class
Mlaznjak represents the actual spacecraft and its physical environment.
It is responsible for:
- spacecraft position,
- spacecraft velocity,
- target position,
- gravity,
- thruster activation,
- interaction with the neural network,
- random exploration,
- training,
- particle-based engine exhaust,
- rendering.
The important physical state variables are:
x, y xboost, yboost target_x, target_y
The spacecraft does not move directly according to the ANN output.
Instead, the thrusters change its velocity:
velocity + thruster acceleration + gravity ↓ new velocity ↓ new position
There is also slight natural damping, so the spacecraft gradually loses velocity when the engines are not compensating for it.
9. Engine Exhaust
Each thruster has its own independent particle system.
For every nozzle, the program maintains arrays containing the particles':
X position Y position X velocity Y velocity speed lifetime radius
When a particular thruster is active, new particles are emitted from that nozzle.
For example:
↑ UP THRUSTER ↑ ● │ │ LEFT ← ● SHIP ● → RIGHT │ ● DOWN THRUSTER ↓
Particles are given an initial velocity in the direction of the corresponding nozzle. Their velocity gradually decreases, their lifetime decreases, and their visual intensity fades.
When a particle reaches the end of its lifetime, it is reset and reused.
Importantly, each thruster operates independently. If the ANN activates only the left thruster, only the left engine emits new particles. Particles already emitted continue moving and fading even after the thruster is switched off.
10. The Main Module
The main module creates the Pygame window and runs the simulation loop.
At every simulation step it approximately performs:
1. Read current spacecraft/target state 2. Feed the state into the ANN 3. Calculate ANN outputs 4. Select ANN or exploratory action 5. Activate the corresponding thrusters 6. Update velocity 7. Apply gravity 8. Update spacecraft position 9. Calculate the new distance to the target 10. If appropriate, perform backpropagation 11. Update exhaust particles 12. Draw spacecraft, target and exhaust
The current prototype also provides keyboard controls for:
SPACE → enable/disable learning R → reset spacecraft G → enable/disable gravity ESC → quit
11. From Experiment to Game
The current program is essentially the core experiment.
The next stage is to turn it into a game.
One idea is to introduce a maze of narrow corridors. The player would then control the target, rather than the spacecraft itself.
That turns the neural network from merely being an implementation detail into a fundamental part of the game's mechanics.
Possible Applications in Other Games
The same principle can be extended well beyond the spacecraft experiment. The important idea is not the particular neural network or the four thrusters, but the concept of learning a controller that develops its own way of performing a task.
Instead of explicitly programming every movement, an ANN can be trained to control an individual game entity or component. Once training is complete, the learned network can be frozen and used as that entity's permanent controller.
1. Naturally moving enemy spacecraft
For example, consider a space-combat game.
Instead of programming an enemy spacecraft with rules such as:
Turn toward the player, accelerate, fire, turn away, brake, repeat.
an ANN could be trained to perform the task of approaching and attacking a target.
The network could receive information such as:
- relative position of the enemy,
- relative velocity,
- distance,
- direction of the enemy's movement,
- the spacecraft's own velocity,
- possibly information about nearby obstacles.
Its outputs could control:
- acceleration,
- rotation,
- individual thrusters,
- weapons,
- evasive maneuvers.
After training, the network could be frozen and used as the enemy's controller.
The important visual difference is that the enemy would not necessarily move along perfectly calculated trajectories. Its movements could contain small variations resulting from the particular network that was trained.
For example:
Enemy A → aggressive approach Enemy B → wide circling movements Enemy C → frequent braking Enemy D → unpredictable evasive maneuvers
All of them could perform essentially the same task, while exhibiting different movement patterns.
This could make AI-controlled characters appear less like deterministic game objects and more like entities with their own movement style.
2. Training different "personalities"
An especially interesting possibility would be to deliberately train many networks independently.
For example:
Same task │ ┌─────────┼─────────┐ ↓ ↓ ↓ ANN A ANN B ANN C ↓ ↓ ↓ Enemy A Enemy B Enemy C
The physical properties and objective could remain identical, while the initial random weights and training experience differ.
The result would be a collection of controllers that solve the same problem in slightly different ways.
This could be used to create a population of enemies without having to manually design a separate movement algorithm for every enemy.
3. Learning individual parts of a vehicle
The same idea could be applied at a completely different level.
Instead of training the entire vehicle as one neural network, individual components could have their own learned controllers.
Imagine a vehicle made from several independently controlled parts:
Vehicle │ ┌────────┼────────┐ ↓ ↓ ↓ Part A Part B Part C ANN A ANN B ANN C
Each component could be trained to perform a particular movement or control task.
For example, in a mechanical vehicle, different controllers could govern:
- individual wheels,
- suspension elements,
- stabilizers,
- robotic arms,
- articulated joints,
- thrusters,
- wings or control surfaces.
Each component could therefore develop its own response characteristics.
The resulting vehicle would not necessarily behave like a perfectly synchronized mathematical machine. Instead, its components could have subtly different responses because each controller had learned its behavior independently.
4. A vehicle assembled from learned components
This could become particularly interesting if the components were generated or trained separately and then assembled.
For example:
Wheel A → learned controller A Wheel B → learned controller B Wheel C → learned controller C Wheel D → learned controller D
Even though all four wheels have identical physical specifications, their controllers could respond slightly differently.
One might react quickly, another more gradually, and another might compensate for disturbances differently.
The overall vehicle could consequently acquire a unique dynamic behavior emerging from the interaction of its independently learned components.
This is conceptually similar to the spacecraft experiment: the interesting behavior does not have to be explicitly programmed. It can emerge from the interaction between:
learned controllers + physics + environment.
5. Beyond movement
The same principle could also be applied to other game behaviors.
A neural controller could learn:
- how an enemy approaches a player,
- how it avoids obstacles,
- how it retreats after being damaged,
- how a flying creature maintains altitude,
- how a robotic arm reaches an object,
- how a vehicle maintains a trajectory,
- how a character balances while moving,
- how a swarm member follows other members,
- how a creature moves through difficult terrain.
In each case, the designer specifies the objective and the environment, rather than explicitly specifying every individual movement.
6. The key idea: learning the process rather than the result
This is perhaps the most interesting generalization of the original experiment.
Traditional game AI often specifies something like:
If X happens, perform action Y.
A learned controller instead attempts to discover:
What sequence of actions tends to produce the desired result?
The resulting behavior can therefore be different even when the desired outcome is the same.
For example, two enemy spacecraft might both successfully reach the same target:
Target ● / \ / \ / \ A → / \ ← B
but their trajectories could be different because their neural controllers have learned different strategies.
This introduces a potentially useful distinction between behavioral identity and simply having different numerical parameters.
The game designer does not necessarily have to tell the player:
"This enemy has a different AI personality."
The player can discover it simply by observing how it moves.
That is one of the most promising aspects of the original spacecraft experiment: the neural network can become a mechanism for generating individual variations in behavior, rather than merely a replacement for conventional game logic.
Python code:
import math
import random
import pygame
# ============================================================
# Reconstruction of the original Java "Mreza" + "Mlaznjak"
# ============================================================
#
# Controls:
# SPACE - toggle learning on/off
# R - reset network and ship
# G - toggle gravity
# ESC - quit
#
# The implementation intentionally stays close to the old Java
# code rather than "improving" the learning algorithm.
# ============================================================
class Mreza:
def __init__(self):
self.red = 10
self.red_ulaz = 4
self.kol = 3
self.beta = 0.1
self.alfa = 1.0
self.bias = 0.0
self.momentum = 0.9
# Java code uses 1-based indexing. We keep an extra row/column
# so that the structure is easy to compare with the original.
self.a = [[0.0 for _ in range(self.kol + 2)]
for _ in range(self.red + 2)]
self.t = [[[0.0 for _ in range(self.red + 2)]
for _ in range(self.kol + 2)]
for _ in range(self.red + 2)]
self.E = [[0.0 for _ in range(self.kol + 2)]
for _ in range(self.red + 2)]
self.tar = [0.0 for _ in range(self.red + 2)]
self.delta_omega = [
[
[[0.0, 0.0, 0.0] for _ in range(self.red + 2)]
for _ in range(self.red + 2)
]
for _ in range(self.red + 2)
]
self.init_t()
def velicina_mreze(self, kolona, redova, redova_ulaz_izlaz):
self.kol = kolona
self.red = redova
self.red_ulaz = redova_ulaz_izlaz
# Recreate arrays with requested dimensions.
self.a = [[0.0 for _ in range(self.kol + 2)]
for _ in range(self.red + 2)]
self.E = [[0.0 for _ in range(self.kol + 2)]
for _ in range(self.red + 2)]
self.tar = [0.0 for _ in range(self.red + 2)]
self.t = [[[0.0 for _ in range(self.red + 2)]
for _ in range(self.kol + 2)]
for _ in range(self.red + 2)]
self.delta_omega = [
[
[[0.0, 0.0, 0.0] for _ in range(self.red + 2)]
for _ in range(self.red + 2)
]
for _ in range(self.red + 2)
]
self.init_t()
def menjaj_betu(self, x):
self.beta = x
def izlaz(self, i):
return self.a[i][self.kol]
def pattern(self, red1, ulaz, izlaz):
self.tar[red1] = izlaz
self.a[red1][1] = ulaz
def init_t(self):
# Original: r.nextDouble() * 0.01
for i in range(1, self.red + 1):
for j in range(1, self.kol + 1):
for k in range(1, self.red + 1):
self.t[i][j][k] = random.random() * 0.01
def sigmoid(self, x):
if x > 700:
x = 700
elif x < -700:
x = -700
return 1.0 / (1.0 + math.exp(x))
def calc(self):
# Same unusual ordering as Java:
# hidden layer(s), then output layer.
for i in range(2, self.kol):
for j in range(1, self.red + 1):
self.racunaj_celiju_napred_bp(i, j)
for j in range(1, self.red_ulaz + 1):
self.racunaj_celiju_napred_bp(self.kol, j)
def racunaj_celiju_napred_bp(self, k, r):
suma = 0.0
if k == 2:
for l in range(1, self.red + 1):
suma += self.a[l][k - 1] * self.t[r][k][l]
else:
for l in range(1, self.red_ulaz + 1):
suma += self.a[l][k - 1] * self.t[r][k][l]
sigma = self.sigmoid(-self.alfa * (suma - self.bias))
self.a[r][k] = sigma
def propagacija_unazad(self):
self.greska = 0.0
for j in range(1, self.red_ulaz + 1):
self.error_output_layer(self.kol, j)
for i in range(self.kol - 1, 0, -1):
for j in range(1, self.red + 1):
self.error_backward_propagation(i, j)
# Correct weights, retaining the original momentum formula.
for i in range(2, self.kol + 1):
for j in range(1, self.red + 1):
for k in range(1, self.red + 1):
d = (
self.beta
* self.E[j][i]
* self.a[k][i - 1]
+ self.momentum
* self.delta_omega[j][i][k][2]
)
self.t[j][i][k] += d
self.delta_omega[j][i][k][2] = d
def error_backward_propagation(self, k, r):
suma = 0.0
if k == self.kol - 1:
for j in range(1, self.red_ulaz + 1):
suma += self.E[j][k + 1] * self.t[j][k + 1][r]
else:
for j in range(1, self.red + 1):
suma += self.E[j][k + 1] * self.t[j][k + 1][r]
self.E[r][k] = (
self.a[r][k]
* (1.0 - self.a[r][k])
* suma
)
def error_output_layer(self, k, r):
self.E[r][k] = (
(self.tar[r] - self.a[r][k])
* self.a[r][k]
* (1.0 - self.a[r][k])
)
self.greska += abs(self.tar[r] - self.a[r][k])
class Mlaznjak:
def __init__(self):
self.learning = True
self.gravity = True
self.width = 800
self.height = 550
self.mreza = Mreza()
self.mreza.menjaj_betu(0.4)
self.mreza.velicina_mreze(3, 18, 5)
try:
self.brod_image = pygame.image.load("ufo_brod.png").convert_alpha()
self.brod_image = pygame.transform.smoothscale(self.brod_image, (86, 86))
except (pygame.error, FileNotFoundError):
self.brod_image = None
self.reset()
def reset(self):
self.xboost = 0.0
self.yboost = 0.0
self.target_x = 200.0
self.target_y = 200.0
self.x = 100.0
self.y = 100.0
self.prev_distance = self.distance()
self.previous_distance_change = 0.0
self.random_left = 0
self.random_right = 0
self.random_up = 0
self.random_down = 0
self.last_outputs = [0.0] * 5
# Four fixed particle matrices, one for each nozzle.
# particle_x[nozzle][i], particle_y[nozzle][i] etc.
# A particle is reset and reused after its life reaches zero.
self.PARTICLE_COUNT = 28
self.particle_x = [[0.0] * self.PARTICLE_COUNT for _ in range(4)]
self.particle_y = [[0.0] * self.PARTICLE_COUNT for _ in range(4)]
self.particle_vx = [[0.0] * self.PARTICLE_COUNT for _ in range(4)]
self.particle_vy = [[0.0] * self.PARTICLE_COUNT for _ in range(4)]
self.particle_life = [[0.0] * self.PARTICLE_COUNT for _ in range(4)]
self.particle_radius = [[0.0] * self.PARTICLE_COUNT for _ in range(4)]
self.particle_speed = [[0.0] * self.PARTICLE_COUNT for _ in range(4)]
self.particle_next = [0, 0, 0, 0]
# Actual thrust selected for the current movement step.
self.active_thrusters = [0.0, 0.0, 0.0, 0.0]
def distance(self):
return math.sqrt(
(self.target_x - self.x) ** 2
+ (self.target_y - self.y) ** 2
)
def sign(self, x):
if x > 0:
return 1
if x == 0:
return 0
return -1
def input_data(self):
# Equivalent of puniInpDva().
self.mreza.pattern(
1,
1.0 if self.x > self.target_x else 0.0,
0.0
)
# Velocity quantization, as in the original Java:
# ((int)(((boost + 1) / 2) * 4)) / 4
def quantize(v):
return int(((v + 1.0) / 2.0) * 4.0) / 4.0
self.mreza.pattern(2, quantize(self.xboost), 0.0)
self.mreza.pattern(
3,
1.0 if self.y > self.target_y else 0.0,
0.0
)
self.mreza.pattern(4, quantize(self.yboost), 0.0)
def random_thrusters(self):
self.random_left = abs(random.randint(-2**31, 2**31 - 1)) % 2
self.random_right = abs(random.randint(-2**31, 2**31 - 1)) % 2
self.random_up = abs(random.randint(-2**31, 2**31 - 1)) % 2
self.random_down = abs(random.randint(-2**31, 2**31 - 1)) % 2
def move(self):
# Close reconstruction of the original pomeriKrug().
self.random_thrusters()
self.xboost *= 0.99
self.yboost *= 0.99
# Original:
# rrxy[1] = sgn(r.nextDouble()- .3)
rrxy = self.sign(random.random() - 0.3)
# Keep the exact action which was actually used. This is important:
# the original stores this in IzlazMreze and uses it as the target
# for backpropagation on the next learning step.
used_ann = False
used_random = False
if rrxy > 0:
if self.mreza.izlaz(5) > 0.5:
used_ann = True
# ANN controls the ship.
self.action_target = [
self.mreza.izlaz(1),
self.mreza.izlaz(2),
self.mreza.izlaz(3),
self.mreza.izlaz(4),
self.mreza.izlaz(5),
]
xboost2 = (
self.mreza.izlaz(1) - self.mreza.izlaz(2)
)
yboost2 = (
self.mreza.izlaz(3) - self.mreza.izlaz(4)
)
else:
# No thrust in this branch, exactly as the old code did
# (xboost2/yboost2 could retain their previous values).
self.action_target = [
self.mreza.izlaz(1),
self.mreza.izlaz(2),
self.mreza.izlaz(3),
self.mreza.izlaz(4),
self.mreza.izlaz(5),
]
xboost2 = getattr(self, "_xboost2", 0.0)
yboost2 = getattr(self, "_yboost2", 0.0)
else:
if random.random() > 0.2:
used_random = True
# Random exploration.
self.action_target = [
float(self.random_left),
float(self.random_right),
float(self.random_up),
float(self.random_down),
1.0,
]
xboost2 = self.random_left - self.random_right
yboost2 = self.random_up - self.random_down
else:
# Again, preserve the old behavior: no new action.
self.action_target = [
float(self.random_left),
float(self.random_right),
float(self.random_up),
float(self.random_down),
1.0,
]
xboost2 = getattr(self, "_xboost2", 0.0)
yboost2 = getattr(self, "_yboost2", 0.0)
self._xboost2 = xboost2
self._yboost2 = yboost2
# The four actual nozzle activations used by this movement step.
# The network outputs are continuous, so their magnitude controls
# the visual amount of exhaust. Random exploration uses 0/1 values.
if used_ann:
self.active_thrusters = [
self.mreza.izlaz(1),
self.mreza.izlaz(2),
self.mreza.izlaz(3),
self.mreza.izlaz(4),
]
elif used_random:
self.active_thrusters = [
float(self.random_left),
float(self.random_right),
float(self.random_up),
float(self.random_down),
]
else:
self.active_thrusters = [0.0, 0.0, 0.0, 0.0]
# Start NEW exhaust particles only for the nozzles that are ON.
# Existing particles are handled separately in update_thruster_particles().
self.emit_thruster_particles()
self.xboost += xboost2 / 5.0
self.yboost += yboost2 / 5.0
if self.gravity:
self.yboost += 0.02
self.xboost = max(-1.0, min(1.0, self.xboost))
self.yboost = max(-1.0, min(1.0, self.yboost))
self.x += self.xboost
self.y += self.yboost
# Original target bounds.
self.target_x = max(0.0, min(400.0, self.target_x))
self.target_y = max(0.0, min(300.0, self.target_y))
def step(self):
distance_before = self.distance()
self.input_data()
self.mreza.calc()
self.move()
distance_after = self.distance()
current_distance_change = distance_after - distance_before
# This is intentionally the original test:
# if ((rasPosle - rasPre) < brzina)
#
# 'brzina' is the previous distance change.
if self.learning:
if current_distance_change < self.previous_distance_change:
self.set_training_target_from_action()
self.mreza.propagacija_unazad()
# In the Java program:
# brzina = rasPosle - rasPre;
self.previous_distance_change = current_distance_change
if distance_after < 14:
self.target_x = 250.0 * random.random() + 30.0
self.target_y = 150.0 * random.random() + 30.0
def set_training_target_from_action(self):
# This corresponds to puniIzlazZaBP(), which feeds the action
# actually selected during the previous movement back as the
# desired output.
for i in range(1, 6):
self.mreza.tar[i] = self.action_target[i - 1]
def emit_thruster_particles(self):
"""Start particles in fixed per-nozzle arrays.
Nozzle order:
0 = left, 1 = right, 2 = up, 3 = down
Particles start at the nozzle, have an initial velocity, then their
velocity gradually decays. When life reaches zero they are reset
and become available for reuse.
"""
# The sprite is approximately 86x86, so these are near its rim.
nozzle_pos = [
(self.x - 38, self.y), # LEFT nozzle
(self.x + 38, self.y), # RIGHT nozzle
(self.x, self.y - 35), # UP nozzle
(self.x, self.y + 35), # DOWN nozzle
]
# Exhaust direction: each jet shoots outward from its own nozzle.
directions = [
(-1.0, 0.0), # LEFT -> left
(1.0, 0.0), # RIGHT -> right
(0.0, -1.0), # UP -> up
(0.0, 1.0), # DOWN -> down
]
for nozzle in range(4):
strength = max(0.0, min(1.0, self.active_thrusters[nozzle]))
# A nozzle emits NEW particles only when that propulsion channel
# is actually ON. Otherwise we do nothing here: already-fired
# particles continue moving, slowing down and fading naturally.
if strength < 0.5:
continue
# Emit one particle each simulation step, with occasional
# additional particles for stronger nozzle activation.
count = 1
if strength > 0.65:
count += 1
if strength > 0.90:
count += 1
for _ in range(count):
i = self.particle_next[nozzle]
self.particle_next[nozzle] = (i + 1) % self.PARTICLE_COUNT
px, py = nozzle_pos[nozzle]
dx, dy = directions[nozzle]
# Slight random spread keeps the stream organic.
spread = 0.30
vx = dx * (2.0 + 2.0 * strength) \
+ (random.random() - 0.5) * spread
vy = dy * (2.0 + 2.0 * strength) \
+ (random.random() - 0.5) * spread
self.particle_x[nozzle][i] = px
self.particle_y[nozzle][i] = py
self.particle_vx[nozzle][i] = vx
self.particle_vy[nozzle][i] = vy
self.particle_speed[nozzle][i] = 1.0
self.particle_life[nozzle][i] = 1.0
self.particle_radius[nozzle][i] = 2.5 + 3.0 * strength
def update_thruster_particles(self):
"""Move, slow and fade every particle; dead ones are reset."""
for nozzle in range(4):
for i in range(self.PARTICLE_COUNT):
life = self.particle_life[nozzle][i]
if life <= 0.0:
continue
self.particle_x[nozzle][i] += self.particle_vx[nozzle][i]
self.particle_y[nozzle][i] += self.particle_vy[nozzle][i]
# Initial speed gradually falls toward zero.
self.particle_vx[nozzle][i] *= 0.94
self.particle_vy[nozzle][i] *= 0.94
self.particle_speed[nozzle][i] *= 0.94
# Color/brightness is controlled by life.
self.particle_life[nozzle][i] -= 0.035
# Particle slowly shrinks too.
self.particle_radius[nozzle][i] *= 0.985
if self.particle_life[nozzle][i] <= 0.0:
# Reset to inactive state. It will be initialized again
# by emit_thruster_particles() when its nozzle fires.
self.particle_life[nozzle][i] = 0.0
self.particle_speed[nozzle][i] = 0.0
def draw_thruster_particles(self, screen):
"""Draw particles with fading alpha and direction-specific color."""
# Different colors make the four streams easy to distinguish.
# These are visual only; they do not affect the physics.
colors = [
(255, 80, 60), # left
(60, 220, 120), # right
(70, 170, 255), # up
(255, 170, 40), # down
]
for nozzle in range(4):
base = colors[nozzle]
for i in range(self.PARTICLE_COUNT):
life = self.particle_life[nozzle][i]
if life <= 0.0:
continue
radius = max(1, int(self.particle_radius[nozzle][i]))
size = radius * 2 + 4
surf = pygame.Surface((size, size), pygame.SRCALPHA)
# Fade all the way to transparent.
alpha = max(0, min(255, int(255 * life)))
# Slight bright core makes the nearest particles read as
# an actual jet rather than four static dots.
pygame.draw.circle(
surf,
(base[0], base[1], base[2], alpha),
(size // 2, size // 2),
radius
)
screen.blit(
surf,
(
int(self.particle_x[nozzle][i] - size / 2),
int(self.particle_y[nozzle][i] - size / 2),
)
)
def draw(self, screen):
screen.fill((250, 250, 250))
# Four independent streams of fading circular exhaust.
self.update_thruster_particles()
self.draw_thruster_particles(screen)
# UFO from the supplied reference image.
if self.brod_image is not None:
rect = self.brod_image.get_rect(
center=(int(self.x), int(self.y))
)
screen.blit(self.brod_image, rect)
else:
# Fallback if ufo_brod.png is not next to the script.
pygame.draw.polygon(
screen,
(30, 90, 180),
[
(int(self.x + 12), int(self.y)),
(int(self.x - 10), int(self.y - 8)),
(int(self.x - 5), int(self.y)),
(int(self.x - 10), int(self.y + 8)),
],
)
# Target.
pygame.draw.rect(
screen,
(220, 50, 50),
(int(self.target_x - 8), int(self.target_y - 8), 16, 16)
)
# Thruster display.
labels = [
("L", self.last_outputs[0]),
("R", self.last_outputs[1]),
("U", self.last_outputs[2]),
("D", self.last_outputs[3]),
("A", self.last_outputs[4]),
]
font = pygame.font.SysFont(None, 22)
y0 = 350
for i, (label, value) in enumerate(labels):
text = font.render(
f"{label}: {value:.2f}", True, (20, 20, 20)
)
screen.blit(text, (20 + i * 75, y0))
status = (
f"LEARNING: {'ON' if self.learning else 'OFF'} "
f"GRAVITY: {'ON' if self.gravity else 'OFF'} "
f"distance: {self.distance():.1f} "
f"velocity: ({self.xboost:.2f}, {self.yboost:.2f})"
)
screen.blit(font.render(status, True, (20, 20, 20)), (20, 385))
screen.blit(
font.render(
"SPACE=learning R=reset G=gravity ESC=quit",
True, (20, 20, 20)
),
(20, 415)
)
def main():
pygame.init()
screen = pygame.display.set_mode((800, 550))
pygame.display.set_caption("Mlaznjak - reconstruction of 2003 ANN experiment")
clock = pygame.time.Clock()
ship = Mlaznjak()
running = True
while running:
for event in pygame.event.get():
if event.type == pygame.QUIT:
running = False
elif event.type == pygame.KEYDOWN:
if event.key == pygame.K_ESCAPE:
running = False
elif event.key == pygame.K_SPACE:
ship.learning = not ship.learning
print(
"Learning:",
"ON" if ship.learning else "OFF"
)
elif event.key == pygame.K_g:
ship.gravity = not ship.gravity
print(
"Gravity:",
"ON" if ship.gravity else "OFF"
)
elif event.key == pygame.K_r:
ship.reset()
print("Ship reset.")
ship.step()
ship.draw(screen)
pygame.display.flip()
clock.tick(160)
pygame.quit()
if __name__ == "__main__":
main()
Additional: the image must be placed in the same folder as the Python code:
No comments:
Post a Comment