Rudra Dutta, Ismail Guvenc, Mihail Sichitiu, Ozgur Ozdemir, and Magreth Mushi
Published: 28 Jun 2023
CTN Issue: June 2023
A note from the editor:
Aerial Experimentation and Research Platform for Advanced Wireless or EARPAW is a first-of-its-kind aerial wireless experimentation platform that aims to accelerate the integration of unmanned aerial vehicles (UAVs) into the national airspace. Aside from being the first and only platform of its kind, it also looks to enable new advanced wireless features for unmanned aerial system (UAS) platforms. Based at North Carolina State University, the platform provides remote access to researchers for flexible and programmable experiments focused on wireless research and programmable vehicles such as UAVs and rovers. In this month’s article, the authors provide a comprehensive introduction to this exciting platform, discuss the capabilities it provides (including a configuration interface implemented as a digital twin to the physical experimentation environment) and tell us how it can be used to achieve its objectives. AERPAW is an ambitious initiative with high potential, we hope you enjoy reading about it and perhaps some of you will find ways to take advantage of its capabilities.
Miguel Dajer, CTN Editor-in-Chief
AERPAW: A National Facility for Wireless and Drone Research
Mihail Sichitiu
Electrical and Computer Engineering Department, North Carolina State University, USA
Introduction
The Aerial Experimentation and Research Platform for Advanced Wireless (AERPAW) is a national wireless research facility located primarily on and around the grounds of North Carolina State University in Raleigh, North Carolina, USA. Funded by the US National Science Foundation under the Platforms for Advanced Wireless Research (PAWR) program, AERPAW is one of four “city-scale” wireless research platforms funded by the NSF under this program. The development and build of the PAWR platforms is overseen by the PAWR Project Office, funded by the NSF for this purpose. Each PAWR platform focuses on enabling cutting-edge research in advanced wireless topics, and focuses on various challenge directions. AERPAW focuses on enabling experimentation at the intersection of advanced wireless technology, and Unmanned Aerial Vehicles (UAVs), or drone, technology. The AERPAW grant was awarded in September 2019, and the project started in 2020. Despite the challenges posed by the Covid-19 pandemic, work on the project progressed, and the AERPAW facility opened to use by the national community of wireless and drone researchers in November 2021, with a small set of core capabilities. Since then, AERPAW has been operating continuously, and has also continued building out the platform, adding both experimental resources as well as features and capabilities. The architecture and design of AERPAW are somewhat unique to accommodate the combination of ambitious goals and stringent constraints. We provide an overview of the main features of AERPAW in this paper. Detailed information about the various topics presented here are available from the AERPAW User Manual [1], the facility website [2], and other representative previous publications by the AERPAW team [3–11].
Positioning
AERPAW is designed to enable experimentation in support of a broad range of wireless research, from waveform design and antenna/channel characterization to wireless networking, protocol optimization, autonomous navigation, and advanced wireless application design. Especially, the confluence of advanced wireless and drone topics – where wireless networks serve drones, but drones also may serve wireless networks, such as network slicing, increasingly small and variable cell size, sidehaul, 3D aerial networking, disaster recovery networks – represent research topics that AERPAW aspires to enable.
However, such a facility is not useful to any researcher if they cannot access it, and AERPAW’s usefulness would be sharply limited if any researcher who wanted to use it for experimentation needed to physically travel to it. Accordingly, the envisioning from AERPAW’s very inception was that it would be remotely accessible, and usable, by a nationwide community of researchers.
Thus, the core requirements of AERPAW are: (i) remote usability, (ii) embedding in the real-world, representing not just realistic but real Radio Frequency (RF) as well as airspace environment, in an extensive outdoor urban/suburban/agricultural setting, and (iii) the flexibility of equipment to explore this RF and airspace environment, and the ability of the researcher-user to control this equipment, to allow defining a broad range of experimental scenarios in support of a broad range of impactful research projects.
AERPAW Resources
To satisfy this ambitious combination of goals, AERPAW uses Software Defined Radios (SDR), and a custom vehicle and vehicle control platform fabricated from off-the-shelf hardware, open-source code, and purpose-built AERPAW software. Thus, the researcher using AERPAW for experimentation has access to programmable radios, and programmable drones, which they can fully program themselves. The substrate of AERPAW is thus composed of a number AERPAW Hardware Nodes (AHNs), each of which comprises multiple SDRs (National Instrument USRPs), and a companion computer to control them. Some AHNs are “Fixed”, mounted on 60-foot towers (or in a few cases on light poles and rooftops), others are “Portable”, battery-powered, possible to position at arbitrary locations, and also of being mounted on UAVs (or their ground-based wheeled counterparts, called “rovers”). The companion computers on the Portable AHNs are also fully capable of programming and controlling the UAVs and rovers. Thus, the researcher using AERPAW for experimentation has access to programmable radios, and programmable drones, which they can fully program themselves. The Fixed AHNs are installed at various places in a 1500-acre research farm owned by NC State University, and the urban environment of the Centennial Campus of NC State University, providing a wide range of real R/F environments.
These hardware nodes are complemented by a full suite of software solutions, developed either completely by the AERPAW team, or built by the AERPAW team on top of available existing open-source software solutions, that enable using the AHNs for a number of complete meaningful sample experiments, which can serve as templates on top of which a researcher-user can build more complex experiments to serve their own research needs. Notable sample experiments use the AHNs to create complete working systems of Software Radio Systems RAN (srsRAN), Open Air Interface (OAI), GNU Radio, python scripts to collect raw in-phase and quadrature (I/Q) samples, pre-planned UAV trajectories (single and multiple vehicles), joint autonomous SDR and UAV control algorithms.
At the same time, AERPAW provides its users with access to some industry-standard real-world commercial equipment. These include Keysight RF sensors, a complete small Ericsson cellular system with base station and core network that are compliant with 4G and 5G standards, and a system of six Terragraph nodes. Most equipment was received as in-kind contributions as part of the AERPAW grant from NSF, from the respective members (Keysight, Ericsson, NI, and others) of the Industry Consortium formed by NSF to support the PAWR program. Rather than using them simply to enable AERPAW internal operations, AERPAW aspires to enable a national community of researchers access to experimentation with these industry-standard equipment. Not only are they real-world equipment; they are installed – like the AHNs – not in the artificial environment of labs, but in the real-world outdoor settings of AERPAW. Again, sample experiments are provided by AERPAW to help users get started with using these resources, and developing their own custom systems.
AERPAW is also currently developing and integrating LoRaWAN systems, and multiple alternate mmWave experimentation systems, into the facility. Of the mmWave systems, some will be composed of industry-standard equipment, and at least one will be realized by a mmWave system designed and developed at NC State University, partly enabled by the AERPAW project. A radar from Fortem Technologies is also expected to become available (Fortem Technologies specializes in airspace awareness, security, and defense for the detection of drones). The added resources and capabilities are planned to become generally available for use by experimenters in the latter half of 2023 and later. Subsequently, in 2024-25, the platform is expected to complete full operationalization and stabilization of the initial build, and shift into full operational mode, while continuously integrating emerging wireless and drone technology on an ongoing basis.
AERPAW Vehicles
To satisfy the goal of fully programmable air mobility, and full autonomy of UAV operation (enabling running an algorithm at the Portable AHN companion computer that can dynamically compute and execute trajectories, without any assistance from any other computer or human, such as a Ground Control Station), while also enabling multiple levels of custom monitoring, alarms, and abort capabilities, the AERPAW team decided early on that custom-fabrication of purpose-built vehicles was necessary for AERPAW.
The first such vehicle to be designed, fabricated, and built was the Large AERPAW Multirotor (LAM) platform, developed at NC State University under the AERPAW project for use as a standardized aerial platform for carrying large modular payloads. This UAV was designed to give the best combination of performance, flexibility, and in house manufacturability and repair. Its theoretical maximum safe lift capacity is 35 kg, a payload capacity of 23 kg with standard batteries (2 x 25 Ah 6S LiHV – 1,140 Wh). It is a hexacopter with 23-inch propellers, flight time of 35 minutes or better with 3 kg payload (47 minutes with no payload), and 20 km Standard Control / Telemetry Range.
Since then, several other AERPAW vehicle models have been developed and proved, and are in simultaneous use in AERPAW, including smaller quad rotors, and ground wheeled robotic vehicles. They are all compatible with Portable AHNs, in terms of mechanical mounting (allowing for easy transfer of the same AHN from one vehicle to another, for different experiments), electrical, electronic, and programming (vehicle command and control). The AERPAW vehicle design has been described in more detail in [6].
The original LAM (LAM6 to indicate hexacopter) continues to be the exemplar for subsequent vehicle models. It runs the well tested and highly featured ArduCopter firmware on its flight controller. The ArduCopter Firmware runs on The Cube autopilot module, one of the most fully featured and well tested autopilots available today. The autopilot module sits on an in-house designed carrier board built to integrate many of the electronics systems on the drone. This increases reliability, saves weight, complexity of wiring, and allows for functionality not available in off-the-shelf solutions. A host of other necessary components, such as GPS receivers, telemetry and manual override radio controllers, complete the LAM6 specification.
The telemetry capabilities included in the vehicle, piped back through the Portal AHN, enables visualizing the operation of the vehicle on the open-source QGroundControl software. The companion computer enables programming the vehicle through a custom software library provided by AERPAW, which is built on top of the well-known robust MAVLink open standard open-source software (MAVLink is a lightweight protocol used extensively for communication between a Ground Control Station (GCS) and Unmanned vehicles, and in the inter-communication of the subsystem of the vehicle).
AERPAW also has some custom air vehicles beyond its own designed and fabricated vehicles. The most notable example is a helikite, capable of staying up several days with a payload exceeding 10 kg.
Operational Dilemma and Solution
Operation of such wireless equipment, and aerial autonomous vehicles, are subject to regulations from the Federal Communications Commission (FCC), and the Federal Aviation Authority (FAA), respectively. As valuable as the experimental equipment, AERPAW is in possession of the appropriate licenses and exemptions required to operate all this equipment. Full details of the innovation zone license granted to AERPAW, the specific experimental licenses, as well as the FAA exemptions, are available from the AERPAW website [2] and User Manual [1]; Figure 2 summarizes the frequency allocations.
AERPAW also has permits and exemptions in place to fly experimental autonomous air vehicles over its Lake Wheeler Road Agricultural Research Facility, where Fixed AHNs are located. A number of pilots with FAA Part 107 certifications are available to act as observers who can take manual control of the drones if necessary (“safety pilots”, as required by current FAA regulations). AERPAW is also currently in the data gathering phase for making applications for exemptions corresponding to § 107.35 – Operation of Multiple Small UAS and § 107.39 – Operation over human beings of the relevant FAA regulations for AERPAW’s custom vehicles.
AERPAW’s critical value comes not only from the radio and UAV equipment it provides, but as much from the facts that (i) these are embedded in real-world outdoor environments, and that (ii) the AERPAW team has the requisite expertise (and resultant FCC/FAA permits/exemptions) to operate this equipment in a safe and compliant manner while performing meaningful experiment.
This, however, poses a practical dilemma for a research facility that aspires to allow unfettered deep access to such equipment. AERPAW’s value comes from its physical characteristics; but to be of practical use to the nationwide community of researchers, it has to be remotely accessible. Thus, access to AERPAW is heavily mediated by computers and the Internet, but AERPAW is not fundamentally a computing facility. The inclusion of commercial equipment, not designed for such delegated use, makes things even more complex.
The modality that AERPAW adopts in order to balance these limitations is one adopted by many physical research facilities, from telescopes and microscopes to wet labs – that of batch execution [7]. To enable researchers to have as realistic an experience as possible, the AERPAW team has built a Virtual Environment [8, 9] that is accessible through a programming environment indistinguishable from that of the Physical Environment of AERPAW, and whose Radio Frequency and airspace characteristics are close digital twins of the Physical Environment. A web-based portal allows an AERPAW User to log on from anywhere on the public Internet. An Experiment can be initiated in AERPAW’s virtual environment entirely through the web portal. The Experimenter accesses AERPAW’s Virtual environment by standard Secure Shell (ssh) sessions, that allow them to view the source code for sample experiments provided by AERPAW, and build on them to create their own custom experiments. The Experimenter can use the Virtual Environment to run emulated sessions of their experiment at intermediate stages of development, and see visualizations of the execution. When development is complete, the Experimenter can submit the experiment for testbed execution.
AERPAW’s platform control and orchestration framework, consisting of a large amount of custom automation [10], as well as human operators, move the exact code developed by the Experimenter, in a containerized fashion, to AHNs in the testbed, and trigger testbed execution. During testbed execution, overseer containers as well as human safety operators monitor the radio and vehicle usage by the Experimenter’s program, and abort the experiment in case of impending unsafe or non-compliant operation [11]. Thus the Experimenter never leaves the comfort of their own office or lab, need not have a pilot’s license, nor worry about the consequence of unintentionally violating FCC or FAA regulations.
At the successful conclusion of the testbed execution, the experiment is returned to the virtual environment, so the experimenter can again log in, and seamlessly view the results gathered by their experiment.
Other modes of AERPAW use, involving more direct access, or more manual operations, are possible, although the canonical self-programmed access methodology we just described makes most efficient use of resources and automation, and is the cheapest to use.
AERPAW Research
AERPAW’S custom-developed vehicles, and software libraries, together with the digital twin of the AERPAW physical environment, makes highly repeatable mobility experiments possible in AERPAW. Because the vehicle trajectories are programmatically determined, running the same experiment multiple times can ensure that the drones follow precisely the same trajectories each time. Since the same program can also control the software defined radios, joint vehicular autonomy and wireless experiments are straightforward to develop in AERPAW.
Since its original Phase 1 opening in 2021, AERPAW has been continually used for active research, not only by the AERPAW team itself, but external users around the country, and even international collaborators. Some have focused on issues lower down the stack, near the physical layer, while others have focused on network protocols, and characterizations. The AERPAW helikite has been deployed to collect long range data not only in the Lake Wheeler area, but also the highly crowded Raleigh downtown area. We have published a number of papers with our findings enabled by AERPAW [12–14], and so have others. All the wireless data collected by AERPAW is made publicly available through the AERPAW website [15].
At this time, AERPAW is actively working on including multiple mmWave technology options in the AERPAW facility, as mentioned before. The AERPAW team is also actively considering including Open Radio Access Network technology in AERPAW, for use by researchers, and/or to serve as an OpenRAN/O-RAN expertise center.
We hope the AERPAW platform continues to generate interest in the wireless and drone research communities, and successfully enable increasing amounts of meaningful research in these areas, in the foreseeable future.
References
- The AERPAW User Manual. (Available: http://go.ncsu.edu/aerpaw-user-manual)
- The AERPAW Website. (Available: https://aerpaw.org/)
- V. Marojevic, I. Guvenc, R. Dutta, M. L. Sichitiu and B. A. Floyd, "Advanced Wireless for Unmanned Aerial Systems: 5G Standardization, Research Challenges, and AERPAW Architecture," in IEEE Vehicular Technology Magazine, vol. 15, no. 2, pp. 22-30, June 2020, doi: 10.1109/MVT.2020.2979494.
- V. Marojevic, I. Guvenc, M. Sichitiu and R. Dutta, "An Experimental Research Platform Architecture for UAS Communications and Networking," 2019 IEEE 90th Vehicular Technology Conference (VTC2019-Fall), Honolulu, HI, USA, 2019, pp. 1-5, doi: 10.1109/VTCFall.2019.8891315.
- M. M. U. Chowdhury et al., "A Taxonomy and Survey on Experimentation Scenarios for Aerial Advanced Wireless Testbed Platforms," 2021 IEEE Aerospace Conference (50100), Big Sky, MT, USA, 2021, pp. 1-20, doi: 10.1109/AERO50100.2021.9438449.
- Mark Funderburk, John Kesler, Keshav Sridhar, Mihail L. Sichitiu, Ismail Guvenc, Rudra Dutta, Thomas Zajkowski, and Vuk Marojevic. 2022. AERPAW Vehicles: Hardware and Software Choices. In Proceedings of the Eighth Workshop on Micro Aerial Vehicle Networks, Systems, and Applications (DroNet '22). Association for Computing Machinery, New York, NY, USA, 37–42. https://doi.org/10.1145/3539493.3539583
- M. Mushi et al., "The AERPAW Experiment Workflow - Considerations for Designing Usage Models for a Computing-supported Physical Research Platform," IEEE INFOCOM 2022 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), New York, NY, USA, 2022, pp. 1-8, doi: 10.1109/INFOCOMWKSHPS54753.2022.9798061.
- Ashwin Panicker, Ozgur Ozdemir, Mihail L. Sichitiu, Ismail Guvenc, Rudra Dutta, Vuk Marojevic, Brian Floyd, “AERPAW emulation overview and preliminary performance evaluation”, Computer Networks, Volume 194, 2021, https://doi.org/10.1016/j.comnet.2021.108083.
- Mihail L. Sichitiu, Ismail Guvenc, Rudra Dutta, Vuk Marojevic, and Brian Floyd. 2020. AERPAW Emulation Overview. In Proceedings of the 14th International Workshop on Wireless Network Testbeds, Experimental evaluation & Characterization (WiNTECH'20). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3411276.3412188
- M. Mushi et al., "The AERPAW Control Framework - Considerations for Resource Control and Orchestration for a Computing-supported Physical Research Platform," 2022 IFIP Networking Conference (IFIP Networking), Catania, Italy, 2022, pp. 1-9, doi: 10.23919/IFIPNetworking55013.2022.9829763.
- T. Samal, R. Dutta, I. Guvenc, M. L. Sichitiu, B. Floyd and T. Zajkowski, "Automating Operator Oversight in an Autonomous, Regulated, Safety-Critical Research Facility," 2022 International Conference on Computer Communications and Networks (ICCCN), Honolulu, HI, USA, 2022, pp. 1-10, doi: 10.1109/ICCCN54977.2022.9868858.
- S. J. Maeng et al., "National Radio Dynamic Zone Concept with Autonomous Aerial and Ground Spectrum Sensors," 2022 IEEE International Conference on Communications Workshops (ICC Workshops), Seoul, Korea, Republic of, 2022, pp. 687-692, doi: 10.1109/ICCWorkshops53468.2022.9814648.
- S. J. Maeng et al., "SDR-Based 5G NR C-Band I/Q Monitoring and Surveillance in Urban Area Using a Helikite," 2023 IEEE International Conference on Industrial Technology (ICIT), Orlando, FL, USA, 2023, pp. 1-6, doi: 10.1109/ICIT58465.2023.10143158.
- S. J. Maeng et al., "Spectrum Activity Monitoring and Analysis for Sub-6 GHz Bands Using a Helikite," 2023 15th International Conference on COMmunication Systems & NETworkS (COMSNETS), Bangalore, India, 2023, pp. 857-862, doi: 10.1109/COMSNETS56262.2023.10041314.
- AERPAW Public Datasets. (Available: https://aerpaw.org/experiments/datasets/)
Statements and opinions given in a work published by the IEEE or the IEEE Communications Society are the expressions of the author(s). Responsibility for the content of published articles rests upon the authors(s), not IEEE nor the IEEE Communications Society.