CI: Cleanup of parameters, change of branch and image naming, remove BuildProxy
MR Summary:
1. Remove merge step from Jenkinsfile-GitLab-Container
Removes the manual git merge step that was previously performed inside
the `RAN-Container-Parent` pipeline before running CI. The
`doGitLabMerge.sh` call and its merge-conflict failure handler are
dropped from the init stage. The `RAN-Local-Repo-Push` stage no longer
wraps triggerSlaveJob in catchError, instead, its failure posts an MR
comment directly on GitLab to report the issue, and abort the follow-up
test stages.
2. Switch to full commit IDs for image and branch naming
Replaces abbreviated 8-character commit hashes with full commit IDs
everywhere they are used to construct Docker image tags and CI branch
names in internal git repository.
3. Remove L2sim proxy build support
For the `RAN-L2-Sim-Test-4G`, the CI now assumes that the required proxy
image is already present on the testing machine. Build proxy step is no
longer needed, since there haven't been any recent updates in the proxy
repo, and the proxyCommit we currently use is outdated anyway.
4. Rename and clean up parameters in Python scripts
Renames legacy ran* / eNB* prefixed attributes to generic names across
all CI Python classes. Also removes duplicate and unused legacy CLI
argument aliases (--eNBRepository, --eNBBranch, --eNBIPAddress, etc.)
from args_parse.py.
|Old name |New name |
|----------------|----------------|
|ranRepository |repository |
|ranBranch |branch |
|ranCommitID |commitID |
|ranAllowMerge |merge |
|ranTargetBranch |targetBranch |
|eNBSourceCodePath|workspace |
Reviewed-by: Robert Schmidt <robert.schmidt@openairinterface.org>
The Iperf_Module() checks that the iperf3 command succeeds successfully,
but in case of failure, does not collect the actual logs. Thus, only a
somewhat unhelpful
Iperf client command failed on X -> Y (return code: ZZZ)
is logged, and it might not be clear why it fails. In this commit,
always try to collect the logs, which might contain the actual error,
and only add the diagnostic message at the end of the Iperf report
message.
Fixes: f02c84fcfc ("CI: enable PCF-based dual-DNN multi-QoS in 25PRB
RFSim")
Signed-off-by: Robert Schmidt <robert.schmidt@openairinterface.org>
Expand the 25PRB RFSim CI scenario to validate two PDU sessions with
PCF-driven QoS rules and deterministic per-flow traffic ports. Update
iperf execution to use `-B`/`-p` from test args.
Changes:
- Update `container_5g_rfsim_u0_25prb.xml` to add dual-session
validation steps, routing setup, and multi-flow UDP iperf
for UL/DL with explicit bind/port arguments.
- Change `cls_oaicitest.py` to parse bind/port from
`iperf_args`, and return a clear error when the iperf client exits non-zero.
- Update `nrue.uicc.2pdu.conf` to configure
session 2 on `openairinterface` with distinct `nssai_sd` values.
- Extend `5g_rfsimulator/mini_nonrf_config.yaml`
with PCF endpoints, dual slices, dual DNN entries, and PCF policy paths,
and disable local PCC rules in SMF.
- Update `5g_rfsimulator_u0_25prb/docker-compose.yaml`
to add the `oai-pcf` service, mount policy directories, and add ext-dn
route for the second UE subnet.
- Add PCF policy data files for this scenario, in :
`policies/policy_decisions/policy_decision.yaml`
`policies/pcc_rules/pcc_rules.yaml`
`policies/qos_data/qos_data.yaml`
About the ci-scripts:
Co-authored-by: Robert Schmidt <robert.schmidt@openairinterface.org>
Signed-off-by: Guido Casati <guido.casati@openairinterface.org>
- all RAN code, CI code, configuration files, dockerfiles, in CSSL v1.0
- all deployment code (openshift, charts, ancillary files like shell
scripts), in MIT
- documentation in CC-BY-4.0
- exceptions might apply and are listed in NOTICE
- there is a new LICENSES folder with all licenses
- CONTRIBUTIONS.md has been updated accordingly
For automated changes based on OAI PL v1.1:
perl -i~ -0pe 's/\/\*.*Licensed to the OpenAirInterface.*openairinterface.org\n#?/\/*\n * SPDX-License-Identifier: LicenseRef-CSSL-1.0\n/s' **/*.{c,h,cpp}
perl -i~ -0pe 's/\/\*.*Licensed to the OpenAirInterface.*openairinterface.org\n#?/\/*\n * SPDX-License-Identifier: LicenseRef-CSSL-1.0\n/s' **/*.ts
perl -i~ -0pe 's/<!--.*Licensed to the OpenAirInterface.*openairinterface.org\n.*-->/<!-- SPDX-License-Identifier: LicenseRef-CSSL-1.0 -->/s' **/*.xml
The rest (cmake, files with missing license, cmake) manually.
- it is spaghetti code: difficult to understand which code paths are
taken
- the only effect seems to be in the UE log analysis function: it might
make the log fail. However, we test that iperf is successful
(Iperf2_Unidir()), and would detect before the log analysis if MBMS
traffic does not work
- the rest is only messages that likely nobody reads
The real reason for this change is to reduce the number of class members
in the ran.py class, to simplify the functioning of the eNB analysis
function (no self) to reduce dependencies on the ran.py class.
For now, test in the following configuration:
- 16 parallel channels
- channel length of 32 and 128
- with 61440 and 122880 samples
- 12 trials per each test (due to CUDA kernel loading delay in first iterations)
- multiple MIMO configurations: 2TX - 2/4/8/32 RX, 2/4/8/32 TX - 2RX
PRS bug fix and CI Integration
- Fix the bug in PRS channel estimation while copying memory to compute the
impulse response
- Integrate PRS testing to CI
Some CI tasks/commands like Iperf(), Ping() get multiple nodes (via XML
parameter "nodes") to potentially run UEs on different nodes (hosts) at
the same time. However, I argue that this is not good:
- we don't actually use this -- where we specify multiple UEs, it's
always "localhost localhost..."
- it is inconsistent, as we typically have a single "node", not nodes,
and there is a possibility to harmonize (see also next commit)
- if we needed it, there would be better ways to achieve the same.
First, for hardware-based UEs, the ci_infra.yaml can specify different
UEs. For simulated UEs, it would be feasible to have multiple XML
steps run in parallel, e.g., in the XML <parallel> tag, which would
clarify that multiple UEs on different hosts run in parallel
- it reduces code.
Hence, make a single <node> for these XML steps.
Before starting a new iperf3 server instance in CI tests, explicitly kill any existing
iperf3 processes bound to the same port. This prevents conflicts or failures caused by
leftover iperf3 servers from previous runs that did not terminate properly or on time.
The cleanup is performed using `pkill` before launching the new server.
Instead of making the function a class member, make it an instance
member because (1) it is more aligned with other functions, notably,
run(), and (2) especially for RemoteCmd() it can be faster, as
successive exec_script() calls do not require a new SSH connection.
CI: Refactor log collection, collect also (COTS) UE logs
Refactor log handling in CI: collect according to currently used XML files. This
makes it homogeneous in the pipelines, and avoids accidental overwriting (as
happens in phytest-timing), as well as harmonizing the log placement
(e.g., no YAML lookup in Ping/Iperf).
Allow to trace and archive UE logs (e.g., Quectel via QLog, Amarisoft UE logs).
By default, no logs will be collected as of this MR, as many files are big.
Various cleanup in Jenkinsfiles, some bugfixes, add unit tests.
- Add helper function to normalize bitrate values to Mbps for calculations.
- Add helper function to extract data from iperf test report.
- Clean up the UDP analysis function, improve test report formatting.
- Fix an issue where Gbps values were incorrectly reported due to inconsistent
unit parsing.
- Update CI unit test report checks to match the new formatting.
As of the parent commit, there is a generic "may_fail" option through
which we could still ignore an error. Thus, command_fail is superfluous,
and avoids some extra code.
Further, in all these cases, we should better be informed about any
error, so let's not use may_fail for these tests.
In various places, e.g., during initialization (before deployment), we
use the name of the CN object, previously including the IP address.
However, the IP might not be available (because the CN is not deployed),
so we would get warnings at this stage. Avoid this by not looking up the
IP.
There is now one place where we have to look up the IP address
separately.
Also, the exception in initialization might be misleading if we used
{self}, because we complain about no node being present, but the {self}
would include a non-existing node name => do not use it.
With the server object to be passed into ping test, it's not necessary
to either hardcode an IP address or provide a template. Instead, always
fill the IP address, which simplies the Ping() code.
Verify that we don't have an IP address/hardcoded by throwing an
assertion.
This is not really used as of now, because most IPs are hardcoded. See
one of the next commits for the use of this.
To make sure that we have the correct IP, there is some code that
verifies that a hardcoded IP and the CN IP would be the same.
Note: need two separate ci_infra.yaml entries, because
- for FlexRIC, the container name is different (is the tutorial CN)
- for RFsim5G E1, we don't have ext-dn => ping UPF
Use the core network class to configure one "end" of these tests. This
will make the tests more generic and predictable, as they are explicitly
configured through two ci_infra.yaml "entities", instead of some
implicit "EPC" IP address given in configuration.
As of this commit, no XML passes this info into the framework, so no XML
step will work. As it is many changes, the following commits will add
the necessary info one by one.
Add a possibility to use an "infrastructure YAML file" to configure both
ends of the iperf2 call through a file.
Fix bugs in Iperf2: at least on my machine, iperf2 start time is logged
at 0.00 not 0.0, so update the regex to match both. Do not overwrite
results if the matched regex (otherwise, the result might be None at the
end) and suppress server output.
Add a unit test to test that this function actually works (on my
machine, with iperf v2.2.1), as well as a unit test for analysis.
It is clearer in CI logs to use absolute paths, and that is also a
"preference" of the new SSH class (cls_cmd.py) in the sense that some
functions don't handle "cd" gracefully (because of underlying Python
library paramiko).
For zip, though, we should change to have nice relative paths in the
archives, so use cd <path> && zip ... to achieve that, instead of
changing to a directory. This way, the path on which we operate is still
visible in the logs in each line.
