Check timing thresholds on both UE and gNB side. If this commit works as
expected, it can replace xml_files/gnb_phytest_usrp_run.xml.
Signed-off-by: Jaroslava Fiedlerova <jaroslava.fiedlerova@openairinterface.org>
Allow XML test cases to specify one or more files with timing stats that
define which container log paths to collect and analyze. AnalyzeRTStatsObject
loops over the provided list, falling back to the default gNB files
(nrL1_stats.log, nrMAC_stats.log) when none are specified.
Assisted-by: Claude:claude-sonnet-4-6
Signed-off-by: Jaroslava Fiedlerova <jaroslava.fiedlerova@openairinterface.org>
Add automated FHI72 4x4 nightly testing
Add configuration files (conf, YAML, XML) for a new pipeline targeting
nightly high-throughput testing with FHI7.2 RUs. This pipeline is
designed to perform daily performance and stability testing using
either:
- the integration branch (to detect regressions in high-throughput
scenarios), or
- the develop branch (to collect up-to-date performance
metrics and observe long-term stability trends)
Details:
- testing with F1 split: DU runs on stonechat, CU runs on OC
- 4x4 MIMO, 4 DL layers layers with VVDN, LiteON, Metanoia, Bennetel550
and Benetel650 FHI7.2 RUs
- T2 card for LDPC processing
- TDD: DDDSU (special: DDDDDDDDGGGGUU) / DDDSU (special: DDDDDDGGGGUUUU)
for Benetel configrations
Reviewed-By: Robert Schmidt <robert.schmidt@openairinterface.org>
UP3 development UE is used since no dedicated CI UE is available for outdoor
testing. CN with PLMN 20899 can be deployed locally on demand, however for
now always-on CN is used (AMF IP address 172.21.6.5).
The gNB is currently configured with pdsch_AntennaPorts_N1 = 2 instead
of 4, as the proper 8x8 antenna configuration triggers an assertion in
polar procedures (see issue 1067). This will be corrected once the assertion
is resolved.
Signed-off-by: Jaroslava Fiedlerova <jaroslava.fiedlerova@openairinterface.org>
Test description:
- FHI7.2 test with Metanoia RU and Quectel module
- 4 DL layers, 2 UL layers
- stonechat as DU host, CU running on OC
- LDPC processing on the T2 card
Signed-off-by: Jaroslava Fiedlerova <jaroslava.fiedlerova@openairinterface.org>
Test description:
- FHI7.2 test with LiteON RU and Quectel module
- 4 DL layers, 2 UL layers
- stonechat as DU host, CU running on OC
- LDPC processing on the T2 card
Signed-off-by: Jaroslava Fiedlerova <jaroslava.fiedlerova@openairinterface.org>
This commit is intended primarily for comparison with CU/DU split deployments,
enabling data collection and performance evaluation across both configurations.
Signed-off-by: Jaroslava Fiedlerova <jaroslava.fiedlerova@openairinterface.org>
Test description:
- FHI7.2 test with VVDN RU and Quectel module
- 4 DL layers, 2 UL layers
- stonechat as DU host, CU running on OC
- LDPC processing on the T2 card
Enable automatic retry of DU deployment to recover in case of following failure
during T2 initialization:
EAL: Unable to reset device! Error: 11 (Resource temporarily unavailable)
EAL: 0000:01:00.0 setup device failed
EAL: Driver cannot attach the device (01:00.0)
EAL: Failed to attach device on primary process
[PHY] T2 card 01:00.0 not found
Assertion (itf->nrLDPC_coding_init() == 0) failed!
In load_nrLDPC_coding_interface() /oai-ran/openair1/PHY/CODING/nrLDPC_coding/nrLDPC_coding_interface_load.c:66
error starting LDPC library ldpc (null)
Signed-off-by: Jaroslava Fiedlerova <jaroslava.fiedlerova@openairinterface.org>
Use taskset with a dynamic range based on the number of detected processors
for OAI and DPDK build. This ensures both DPDK and OAI build steps use all
available CPU cores on the machine.
Build with XRAN K release
Signed-off-by: Jaroslava Fiedlerova <jaroslava.fiedlerova@openairinterface.org>
"Create new Workspace for server 0/1" no longer makes sense now that
workspace creation is not tied to a specific server index.
Signed-off-by: Jaroslava Fiedlerova <jaroslava.fiedlerova@openairinterface.org>
Remove the `BuildProxy` method, `Build_Proxy` action, `proxyCommit`
attribute, and all associated XML configuration and unit tests.
The oai-lte-5g-multi-ue-proxy repository has not received any updates
in over two years, and the proxy commit currently used for the build
is outdated as well. Although the proxy image is still required by
RAN-L2-Sim-Test-4G and must remain available on the test machine,
building it as part of the CI pipeline is dropped in this commit.
Instead, the CI will now assume that the corresponding proxy image is
already present as an external dependency, similarly to how CN images
are handled.
Signed-off-by: Jaroslava Fiedlerova <jaroslava.fiedlerova@openairinterface.org>
- new configuration for PLMN 00105, 40MHz and 100 MHz added on AmariUE
- helm charts for OC CN with PLMN 00105 added to cacofonix, new CN added
to ci_infra.yaml, IP address range of the new CN: 172.21.6.116-118
- test with 1 AmariUE (to be extended in future MR), with SISO
Signed-off-by: Jaroslava Fiedlerova <jaroslava.fiedlerova@openairinterface.org>
Running container in privileged mode and host network mode is needed
for running fhi72 gNB with M-plane
Signed-off-by: Jaroslava Fiedlerova <jaroslava.fiedlerova@openairinterface.org>
Running container in privileged mode and host network mode is needed
for running fhi72 gNB with M-plane
Signed-off-by: Jaroslava Fiedlerova <jaroslava.fiedlerova@openairinterface.org>
After attach, run UL ping so N3 UL traffic hits the UPF
before the first DL ping. That triggers UL PDR classification
in the UPF and avoids DL GTP-U using the default QFI.
Signed-off-by: Guido Casati <guido.casati@openairinterface.org>
Expand the 25PRB RFSim CI scenario to validate two PDU sessions with
PCF-driven QoS rules and deterministic per-flow traffic ports. Update
iperf execution to use `-B`/`-p` from test args.
Changes:
- Update `container_5g_rfsim_u0_25prb.xml` to add dual-session
validation steps, routing setup, and multi-flow UDP iperf
for UL/DL with explicit bind/port arguments.
- Change `cls_oaicitest.py` to parse bind/port from
`iperf_args`, and return a clear error when the iperf client exits non-zero.
- Update `nrue.uicc.2pdu.conf` to configure
session 2 on `openairinterface` with distinct `nssai_sd` values.
- Extend `5g_rfsimulator/mini_nonrf_config.yaml`
with PCF endpoints, dual slices, dual DNN entries, and PCF policy paths,
and disable local PCC rules in SMF.
- Update `5g_rfsimulator_u0_25prb/docker-compose.yaml`
to add the `oai-pcf` service, mount policy directories, and add ext-dn
route for the second UE subnet.
- Add PCF policy data files for this scenario, in :
`policies/policy_decisions/policy_decision.yaml`
`policies/pcc_rules/pcc_rules.yaml`
`policies/qos_data/qos_data.yaml`
About the ci-scripts:
Co-authored-by: Robert Schmidt <robert.schmidt@openairinterface.org>
Signed-off-by: Guido Casati <guido.casati@openairinterface.org>
CI: Various CI adjustments
- RAN-SA-FHI72-CN5G: activate/inactive carriers on VVDN RU - to check if
it improves reliability and stability of the pipeline
- RAN-SA-FHI72-CN5G: update CN with new IP range - avoid IP address
overlap with CN used in RAN-SA-AERIAL-XXX pipelines
- RAN-SA-B200-Module-SABOX-Container: test with deltaMCS in SC-FDMA test
case
- RAN-NSA-B200-Module-LTEBOX-Container: adjust PRACH DTX threshold to
reduce number of false RA attempts on eNB
- remove retx check in RAN-SA-Multi-Antenna-CN5G and RAN-SA-AERIAL-CN5G -
to avoid false CI failure caused by unsuccessful retx check of
interfering UE
- adjust RAN-SA-Multi-Antenna-CN5G and RAN-SA-AERIAL-CN5G configs to
improve UL throughput
Add vvdn-activate-carriers.sh to verify RU PTP synchronization and
configuration. Activates carriers on VVDN RU by editing sysrepo running
datastore (INACTIVE->ACTIVE).
Add vvdn-inactive-carriers.sh to reverse the operation, setting all
carriers back to INACTIVE via sysrepocfg.
Retx check frequently fails in OTA pipelines, particularly for interfering
UEs attempting to connect to the cell. This leads to job failures in OTA
pipelines.
Remove the retx check from Multi-Antenna and Aerial pipelines to improve
CI stability and avoid false failures. In these pipelines, iperf test
already validates correct funtionality, making the retx check redundant.
We don't enforce cppcheck through the CI, although it's there since
years. It runs on Ubuntu 18/20, so it's old. For folks, it's likely not
discoverable on how to run it locally. Let's make a fresh start.
This removes cppcheck from all CI-related code. Instead, it adds it
under tools/cppcheck/, including documentation on how to run it locally,
bare-matel or in docker.
Signed-off-by: Robert Schmidt <robert.schmidt@openairinterface.org>
- remove test_multipath and test_noise from ChanelSim tests of
RAN-Channel-Simulation pipeline - will be executed together with the
other CUDA-enabled unit tests
- enable configurable dockerfile, runtime-opt, and ctest-opt parameters in
the XML file for the Build_Run_Tests step
- pass runtime options to `docker run` to support ctest execution with CUDA
- allow additional ctest options for selective test execution and labeling
- all RAN code, CI code, configuration files, dockerfiles, in CSSL v1.0
- all deployment code (openshift, charts, ancillary files like shell
scripts), in MIT
- documentation in CC-BY-4.0
- exceptions might apply and are listed in NOTICE
- there is a new LICENSES folder with all licenses
- CONTRIBUTIONS.md has been updated accordingly
For automated changes based on OAI PL v1.1:
perl -i~ -0pe 's/\/\*.*Licensed to the OpenAirInterface.*openairinterface.org\n#?/\/*\n * SPDX-License-Identifier: LicenseRef-CSSL-1.0\n/s' **/*.{c,h,cpp}
perl -i~ -0pe 's/\/\*.*Licensed to the OpenAirInterface.*openairinterface.org\n#?/\/*\n * SPDX-License-Identifier: LicenseRef-CSSL-1.0\n/s' **/*.ts
perl -i~ -0pe 's/<!--.*Licensed to the OpenAirInterface.*openairinterface.org\n.*-->/<!-- SPDX-License-Identifier: LicenseRef-CSSL-1.0 -->/s' **/*.xml
The rest (cmake, files with missing license, cmake) manually.
Fix Delta MCS mode, reimplement UL power control
This change set fixes the "delta-MCS" mode, and reimplements the UL power
control loops at gNB, i.e., for PUSCH and PUCCH. It further adds new T traces
to log PUSCH/PUCCH power infrmation, and cleans up code.
The "delta-MCS" mode in OAI refers to configuring a UE with deltaMCS in the
PUSCH-PowerControl IE (see TS 38.331). In this mode, the UE adjusts the PUSCH
power for transmissions to take into account for MCS (see TS 38.213 Sec. 7.1).
In other words, the target SNR can be set to a somewhat low value, and the UE
will still use the right power for UL transmissions. Note that the spec only
foresees this adjustment of UL power when using one layer. In contrast, in the
"normal" mode, the gNB sends TPC commands to keep a UE at a fixed target SNR
(default: 20dB), regardless of the MCS or number of layers. Future work is
planned to completely remove this fixed target SNR, which is not done in this
MR yet.
Further, there is a new power control loop implementation. Prior to this MR, on
each UL transmissions, the gNB uses the SNR to decide about TPC sent to a UE in
the next transmissions. The new implementation relies on averaging the SNR, and
a "TPC in flight" average to account for the slow reaction of the average SNR.
The sum of both quantities is the current SNR that is used by the UE. See the
commit messages or documentation for more information.
Further cleanup to group data (e.g., power control configuration, refactoring of
code to save space, ...) is done. Two new T traces GNB_MAC_PUSCH_POWER_CONTROL
and GNB_MAC_PUCCH_POWER_CONTROL are added; documentation exists to explain how
to graphically plot corresponding graphs.
Do not limit DL power, because it is not done in the other pipelines
either. Set the UL target power high, as this increases the achievable
throughput high (PUCCH can be normal). Finally, increase BLER thresholds
to improve the MCS a bit.
* Option to build ldpc_cuda was remaining in the build script
* Build script option to build ldpc_cuda was still used in CI
* Mentions of ldpc_cuda were remaining in the documentation
Note: In the documentation, the example of ldpctest usage that was
loading ldpc_cuda was replaced by an example in which it is loading
ldpc_orig even though this is pointless as it makes ldpctest compare
ldpc_orig with ldpc_orig. On the other hand, it is the only library
other than the default ldpc that uses the former interface.
CI: Add FHI7.2 phytest pipeline
This MR consists of multiple changes related to FHI7.2 setups in CI.
- remove unused fhi72 configurations in CI
- introduce dpdk-init service in fhi72 deployments to setup sriov on the machine
- update phytest fhi72 configurations for running on bulbul
- add phytest fhi72 to RAN-SA-FHI72-CN5G pipeline
- Introduce dpdk-init service for SR-IOV initialization within the container
- Execute script to enable low-latency RT performance (restored after tests)
When running phytest with FHI7.2 in Docker, two services are deployed:
dpdk-init and oai-gnb. Previously, AnalyzeRTStatsObject selected the
first deployed service for RT stats analysis.
Allow to explicitly select the service used for RT stats analysis instead
of relying on the first deployed service.
CI: rework log analysis to simplify and allow new use cases
The CI has some log analysis, but it is unmaintainable (huge functions
that mix eNB/gNB or lteUE/nrUE, are not tested, most of it is not used
or useful...) or limited to specific use cases (the timing-phytest time
analysis is hardcoded to source builds instead of also handling analysis
when running from docker images). This MR reimplements both functions.
For the log analysis, the CI team decided to throw away the old
analysis, and reimplement a mechanism that checks the couple of things
we care for.
- Allow to analyze phy-test timing analysis from docker files, and add a
unit test
- Rework existing log analysis to remove useless checks, and make it
more composable
* The undeployment steps (Undeploy_Object(), Terminate_eNB()) take an
<analysis> XML object which then either has a <services> subelement
with check that are whitespace delimited (similar to how other xml
task steps have <services>). For checks that contain whitespace,
they can use <service> (singular) that encodes a single check, such
that e.g., arguments could include spaces.
* Each check is of form service=check[=arguments], where service is
the docker services, check is a checking function (implemented as a
class with run(file, options) function inside cls_loganalysis.py,
and options arguments after the second = that is to be read by run()
(basically, options will contain arguments that can be freeform)
* these definitions can occur multiple times, e.g., it could be
serviceA=Check1 and serviceA=Check2=Args and it will execute both
Check1 and Check2
* if a service is declared, it will execute alse the Default check
which currently checks for an assertion
* Implemented (and inspired from the existing CI analysis) are:
- Default check to check for assertions (will be extended, to e.g.,
address/undefined behavior sanitizer analysis)
- RetxCheck: retransmission check as used in the CI
- LastLineContains to check the last line containes a specific
string
- ContainsString to check a specific string appears in logs
- EndsWithBye: check that towards the end (but not necessarily the
last line), there is a Bye. string (a workaround since we don't
always terminate with 0)
- There is a plan to use return codes after stopping in docker, but most
executables do not end with 0 most of the time, such that we needed to
abandon this plan.
- Additional fixes (see commit messages) and delete unused code
