Abstract
The increase of IoT devices and the emergence of smart cities have revolutionized urban development, offering numerous benefits while addressing environmental concerns. This has caused an increase in the usage of IoT frameworks, and the need for efficient architecture and standardized ontology is imminent. In this regard, we propose a distributed, multi-layered data platform architecture comprising the Data Monitoring Layer (DML), Data Storage Layer (DSL), Data Enhancement Layer (DEnL), and Data Exchange Layer (DEL). Our architecture achieves interoperability, facilitates data transfer between nodes, enables telemetry data retrieval, and ensures cross-platform and cross-device compatibility. It addresses the challenges of handling increased sensor data and user demands by providing high throughput and scalability support. We investigated Smart City Living Lab at IIIT Hyderabad an existing large-scale system deployed within a 66-acre campus. This system consists of 291 nodes. By studying this deployed system, we were able to gather valuable real-world data, allowing us to analyze the challenges and potential solutions related to data architecture. Our results show improvements of up to 41.23\% in throughput and a decrease in latency by 29.19\% for data insertion from the sensor nodes. The retrieval by the data client gives an increase of over 800\% in both throughput and number of requests through DENL. These metrics are compared to a centralised data platform architecture. We conclude by discussing the implications of our findings and suggesting future work.