kimi-k3-in-c
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
10 repos · 29 papers · 56 scholar
Showing 95 of 95 items
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
Collection of AWESOME vision-language models for vision tasks
Hands-on LLM algorithms and systems with PyTorch, Triton & CUDA.《大模型算法与系统教程》:模型架构与训练、后训练、推理加速、性能与算子优化,配套 Notebook 练习、答案解析、测试与实验。
SNN vs Dense MLP benchmark: 98.3% sparsity, 122× energy advantage on neuromorphic hardware. Custom LIF neurons in PyTorch on RTX 5060. 10-run statistical analysis.
A wearable affective computing system combining in-ear EEG and facial EMG sensors with custom analog front-end circuits for real-time emotion recognition.
SNN training with Surrogate Gradients in pure PyTorch. 95.0% accuracy, 78% sparsity, 43.9x energy advantage on neuromorphic hardware. Timestep scaling + Pareto front analysis on MNIST.
Sourced comparison of LLM inference chip architectures: GPU, wafer-scale, LPU, transformer ASIC, model-on-chip and edge compute-in-memory. Tokens/s figures with who measured them.
Contains the code for examples offered in the paper
Wafer-scale accelerators offer massive compute capacity through hundreds of thousands of processing elements (PEs) interconnected across a single wafer, but realizing this potential requires efficient application mapping. Their scale makes programming these systems and mapping applications onto them challenging. Existing approaches rely on template-based mappings to maintain productivity, but these templates restrict the mapping design space. In particular, they impose fixed geometric layouts over the PE grid, often leaving compute and memory resources underutilized. In this work, we present WaferTetris, a system that removes these geometric restrictions through freeform mapping. Its Tetris-like strategy flexibly packs computation across the wafer, improving PE utilization and usable memory capacity while preserving PE-task code. We develop a recurrence-based throughput model that captures branch-and-join dependencies to guide spatial parallelism, and an architecture model with per-color gates that enable existing FPGA place-and-route algorithms to enforce WSE routing constraints. We evaluate WaferTetris on Cerebras WSE-3 using LLM decoding and scientific stencil benchmarks. For LLM decoding, it achieves up to $4.42\times$ the system throughput of a template-based automatic mapping baseline on the same fabric size. Compared with the manually optimized WaferLLM, WaferTetris supports $1.8\times$ the context length for LLaMA3.1-8B (80k versus 44k), while delivering equal or higher per-user throughput at matched context lengths, up to $1.10\times$.
Long-context LLMs must retain growing histories, but full attention makes each new token increasingly expensive. Inference-time learning (ITL) offers another approach. The model keeps training on the context during inference, retaining what it learns in its weights. Recent work shows that ITL can match full-context attention quality while attending only to a recent fixed-size window. This bounds attention costs but adds Learning (backward computation and weight updates) alongside Prediction (the forward pass) costs. Inference accelerators assume fixed weights, while training accelerators target complete training steps. This work identifies two opportunities. Attention's non-matrix operations leave enough idle matrix capacity for Learning, while selected layers can tolerate delayed weight updates with little quality loss. We exploit both with Learning While Serving (LWS), an inference accelerator architecture that shares compute hardware between latency-critical Prediction and deadline-flexible Learning. Hardware-managed per-layer weight versioning allows Learning to publish updates while in-flight computations read older versions. Resumable reduction blocks let Learning kernels span fragmented idle windows and yield to Prediction, while dedicated softmax-gradient units relieve vector contention. On modeled datacenter and edge NPUs, LWS hides Learning within Prediction, sustaining Prediction-only throughput while preserving 99.3% of fully fresh Learning's quality benefit for TTT-E2E 3B at 128K tokens. Across nine models, LWS achieves 1.6x the throughput of serial execution and 1.3x that of the best prior co-execution architecture. RTL synthesis shows that LWS adds 1.5% to NPU core area without another MAC array. By efficiently turning idle inference capacity into learning capacity, LWS establishes inference-time learning as a first-class workload for future LLM accelerators.
Linear-attention language models replace a growing key-value cache with a recurrent state of fixed size. On an edge FPGA, this state can exceed the on-chip memory and must be paged to external memory. This paper asks whether such paging dominates the cost of decoding (generating tokens). We built a hardware tile that computes one complete layer of a custom 369-million-parameter hybrid Gated DeltaNet model on an Agilex 5 FPGA and pages its recurrent state. The model state needs 6.2 MiB, whereas the device has 0.87 MiB of block memory. Paging adds 26.0 MB of memory traffic per decoded token, at most 11.2 percent of the calculated total. On the board at 75 MHz, with synthetic weights, one token takes 11.15 ms in this layer. The matrix-vector products with the weights take about 62 percent of this time and the state phase, including the paging, 13 percent. The tile is one node of a multicast mesh of tri-mode processing elements. We measure the error of the fixed-point arithmetic against floating point and check that the hardware agrees bit for bit with a fixed-point reference. In simulation, the state logic agrees with the reference on all 18 recurrent layers. On the board, six builds pass all 30 runs of 64 tokens and two runs of 1,024 tokens. For this model and memory interface, the delivery of the weights limits the decoding performance, and the recurrent state does not. State paging is therefore practical for such a model on an edge FPGA.
Stack-based reconvergence enables efficient SIMT execution with a shared warp program counter and compact control-flow state. However, its fixed serialization of divergent paths can deadlock programs that rely on intra-warp synchronization: a spinning subgroup may prevent a resource-holding sibling from executing. Independent Thread Scheduling (ITS) enables finer-grained progress through per-thread execution state, but introduces additional hardware cost. We present SIMTurn, a lightweight compiler-hardware mechanism that enables forward progress among divergent subgroups while retaining stack-based reconvergence. SIMTurn combines a compiler-inserted yield instruction at potentially blocking loops with a bounded pool of resumable subgroups. The scheduler allows a waiting subgroup to yield to a sibling that can make progress, without requiring independent program counters for every thread. We implement SIMTurn in the compiler, simulator, and RTL on the open-source Vortex GPGPU and evaluate it against baseline IPDOM and per-thread ITS schemes. Across the evaluated GPU thread configurations, SIMTurn introduces only 17-24% scheduler-area overhead relative to IPDOM, compared with 39-91% for ITS, and stays within 8% of IPDOM on workloads that never need cross-subgroup progress. On kernels with genuine cross-subgroup parallelism, SIMTurn tracks ITS closely and is up to 2.7x faster than a software-serialized IPDOM baseline.
Next-generation wireless research relies on software-defined radios (SDRs) with more channels and larger bandwidth. The USRP family is one of the most widely used SDR platforms, and much of the wireless software ecosystem is built on top of the USRP hardware driver (UHD). However, each USRP model comes with a fixed channel count, bandwidth, and streaming capability. Off-the-shelf radio frequency system-on-chip (RFSoC) boards, on the other hand, integrate up to 16 analog-to-digital converters (ADCs) and 16 digital-to-analog converters (DACs) operating at several gigasamples per second (GS/s), but existing UHD applications cannot directly use them. We present uhd-rfsoc, an open-source FPGA design and host software stack that turns these boards, including the 16-channel ZCU216 and the RFSoC 4x2, into standard UHD devices without modifying UHD. Through standard UHD calls, applications can configure the carrier frequency, channel count, the per-channel I/Q sample rate, and the wire format. uhd-rfsoc combines timestamped, flow-controlled sample transport in the FPGA with data plane development kit (DPDK) packet processing and vectorized sample conversion on x86 and Arm hosts. The built bitstreams for a number of RFSoC board configurations with varying RF data converter rates, channel counts, I/Q sample rates, and wire formats all successfully pass UHD's built-in streaming benchmarks. We further demonstrate compatibility with unmodified GNU Radio flowgraphs, a custom GPU-accelerated transceiver digital signal processing pipeline, as well as the OCUDU and Duranta 5G radio access network stacks using both software and commodity user equipment.
Physical design automation of very large-scale integrated circuits (VLSI) has made significant strides, progressing from single-stage placement and routing toward increasingly integrated full-flow implementation. Yet, existing approaches struggle with the vast design space optimization (DSO) arising from diverse technology nodes, circuit designs, and varied constraints. The substantial runtime of physical-design tools further exacerbates this challenge, leaving industrial workflows heavily dependent on experienced engineers to navigate the design space and achieve power, performance, and area (PPA) targets, incurring significant engineering effort. We argue that large language models (LLMs) with strong reasoning capabilities, enriched with distilled human design expertise, offer a promising path toward efficient and automated DSO. Unlike traditional algorithms, LLMs can interpret complex design contexts and naturally incorporate human expertise expressed in natural language for reasoning, acting, observing, and adapting throughout the design process, a capability that conventional optimization methods cannot exploit. To this end, we present OpenLayout, an agentic platform that leverages multi-agent collaboration with embedded design knowledge to enable self-iterative DSO. We further introduce OpenLayout-Bench, a benchmark targeting advanced technology nodes with commercial EDA tools to systematically evaluate physical design automation capabilities of LLMs. Experimental results demonstrate that OpenLayout achieves state-of-the-art performance, with approximately 20% accuracy improvement over baseline methods. Our work is open-sourced at https://github.com/fenggev56/OpenLayout
Growing visual data overwhelms storage on resource-constrained edge devices, making compression essential. Traditional codecs offer inadequate rate-distortion performance, motivating the exploration of diffusion codecs. Diffusion codecs face two limitations: (1) At the algorithm level, diffusion decoders preserve perceptual quality under extreme compression, but require substantial computation. (2) At the architecture level, in NPU-centered devices, neural codecs occupy the NPU, interfering with other AI applications. Meanwhile, the complex codec pipeline incurs long I/O paths, leading to additional overhead and energy cost. To address these issues, we propose DiffusionSSD, a cross-level near-data processing framework for extreme visual data compression on edge devices, using an algorithm-architecture co-design method. Specifically, (1) at algorithm level, we present Dual-stage Diffusion, a lightweight solution tailored for diffusion codecs that employs patchification and single-step diffusion, reducing computation cost while preserving perceptual quality; (2) at architecture level, we introduce a Cross-level Near Data Processing architecture that integrates the proposed computational universal flash storage (cUFS) and LPDDR-PIM, enabling codecs to be entirely offloaded to data side while reducing visual-data movement and avoiding SoC redesign; (3) at system level, combining both the algorithm and the underlying architecture, we propose a Patch-Workload co-aware scheduling scheme to enable more efficient system-level execution. Experiments show that Dual-stage Diffusion maintains comparable perceptual quality while reducing computation cost by up to 92.7%. Compared with NPU+DRAM and near-data-processing design such as SmartSSD, cUFS+PIM improves coding throughput by 2.32 times and 2.91 times on average, respectively.
Mixture-of-Experts (MoE) architectures activate only a subset of experts for each token, allowing large language models (LLMs) to scale parameter capacity without a proportional increase in computation per token. However, intermediate data movement and uneven expert loads limit the efficiency of MoE inference on TPUs. To address these bottlenecks, we introduce Fuse-All MoE, a TPU kernel design that jointly optimizes expert computation, output communication, and load balancing. The kernel fuses token collection, expert projections, gated activation, and output writes, retaining expert computation intermediates on chip. Fixed token-slot destinations allow scheduling and expert placement to change while preserving routing semantics. In prefill-decode (PD) disaggregated serving, Fuse-All MoE balances EP16 prefill workloads through scheduling and expert redistribution, while symmetric EP8TP2 shares active expert computation between the two TensorCore devices within each chip during decode. Our experiments show that Fuse-All MoE reduces kernel runtime for a measured skewed EP16 prefill workload from 2.3x to 1.09x the same fixed reference time for a balanced workload. Across 32-512 decode tokens, symmetric EP8TP2 reduces kernel latency by up to 21.3% relative to fused EP16. With accompanying serving stack optimizations, the integrated system improves serving throughput by up to 34.5% over the Megablox EP16 baseline.
Edge AI increasingly relies on specialized accelerators for low-latency inference under tight resource and energy constraints, yet their efficiency remains limited by data movement. Prior work, CHARM, mitigates this overhead by overlapping data movement with computation. We observe that in-memory computing can further supplement tensor processing as a parallel accelerator while eliminating certain data movements entirely. We present CHARMING, a heterogeneous edge AI architecture that uses accuracy- and quantization-aware partitioning to distribute and concurrently execute inference workloads across tensor processing and in-memory computation. CHARMING partitions attention heads between an AIE array and an in-DRAM compute unit, balancing execution according to model architecture, sequence length, and precision while selecting heads to minimize accuracy degradation. CHARMING also integrates quantization into the AIE projection stage to further reduce data movement. We validate CHARMING end-to-end in hardware on AMD Versal VCK5000 and Alveo U200 platforms and evaluate LLM and video-generation models across multiple precisions using measured hardware results and simulation of the latest in-memory compute hardware. CHARMING improves prefill performance by up to 23.6 percent and decode performance by up to 44 percent over AIE-only acceleration. With int4 x int4 fixed-point in-memory computation, CHARMING reduces per-layer energy by up to 7.6 percent and improves energy-delay product by up to 24.7 percent, while maintaining accuracy similar to a 32-bit floating-point baseline.
Kernel optimization is essential for efficient large language model (LLM) training and inference on Ascend NPUs. Existing Ascend kernel benchmarks primarily focus on kernel generation, with less emphasis on the ability of agents to optimize existing implementations under input configurations drawn from real workloads. To address this requirement, we introduce AscendGYM, a benchmark whose initial suite comprises 48 kernels across three difficulty levels. Each kernel is paired with test cases whose data types and tensor shapes are collected from real-world training and inference tasks. Consequently, performance improvements on AscendGYM map directly to the corresponding workloads. In terms of usability, two evaluation modes are supported in AscendGYM: model evaluation and model-harness evaluation. The kernel optimization capabilities of both trained models and custom agents are allowed to be measured. It provides a complete execution environment that closely reproduces how Ascend kernels are invoked in practical pipelines. For model evaluation, it additionally includes a built-in harness equipped with search orchestration and experience accumulation. Finally, we propose a new metric, the Speedup-Token Score (STS), which quantifies performance gains relative to token cost. By jointly considering speedup and cost, STS enables the assessment of practical kernel optimization capability of a model or harness under an acceptable budget. AscendGYM is available at https://github.com/SLAI-AITP/AscendGYM.
Recent work shows that large language models (LLMs) can operate directly on the parameters of other neural networks. They can read what a network was trained to do, fix a network that fails, and write working networks from scratch, all without gradient descent. This capability has been termed Weight-Space Programming (WSP). Exploring this new paradigm systematically requires a robust benchmark. We introduce WEIGHTBENCH, a benchmark that evaluates LLMs of increasing capability on 42 WSP tasks across five tracks: reading, fixing and writing weights, recognizing edits that are impossible, and data-free compression. The networks whose weights the LLMs read and edit range from 2K-parameter control policies to 7B-parameter language models. Experimental benchmark results show that frontier agents can already read small networks and write working control policies without ever running them, but cannot yet patch language models, compress them without data, or tell an impossible edit from a hard one. On all 42 tasks, Claude Opus 5.5 reaches a macro score of 0.535 (Sonnet 5.5 0.466, Haiku 5.5 0.402, DeepSeek v4-flash 0.215); on the 38 tasks it was run on, GPT-6.1-sol reaches 0.568 against Opus's 0.546. Code, the task registry, our OS-level sandbox harness, and the live leaderboard are publicly available at https://github.com/Aayush-Marasini/WeightBench.
Screen history can serve as searchable visual memory for people and agents, but unlike an offline corpus, it must be encoded and indexed as each screenshot arrives. OCR-based indexing is fast but loses visual information; visual encoding takes over two seconds. Yet screens change incrementally: most pixels repeat within and across screenshots. This raises a question: do visual encoders need to recompute their representations from scratch for every new screenshot? We present StreamEmbed, a system for incremental visual embedding that exploits this redundancy. Guided solely by pixel differences, it combines spatial token pruning, which skips blocks that repeat a neighbor, with temporal KV reuse, which reuses cached keys and values for blocks unchanged across screenshots. A small learned adapter compensates for pruning and reuse to preserve fidelity to full-screenshot embeddings. On ScreenMemEval, our benchmark of 10,426 questions over 184 hours of screen histories from two humans and a computer-use agent, StreamEmbed sets a new accuracy-latency Pareto frontier on an M2 Pro GPU: it encodes up to 3.7x faster than full-screenshot encoding without sacrificing accuracy and improves accuracy over OCR-based indexing by up to 6.3 Recall@10 points at matched latency. A memory-efficient vector index stores a simulated year of screenshots in 0.13 GB of RAM. Together, these results make real-time indexing of screen history in its native visual form practical, laying a foundation for fully local visual assistants.
Barron et al. (2026) recently identified numerical issues in the softmax and soft cross-entropy loss. We show that similar issues are present in standard divergence operations between categorical distributions. This includes the forward and reverse Kullback-Leibler (KL) divergence and generalized Jensen-Shannon divergence (JSD), as well as Monte Carlo KL estimates and entropy. When the distributions are nearly equal, rounding and cancellation issues affect the forward and gradient computations. These issues can affect settings using divergences in optimization objectives, such as knowledge distillation, reinforcement learning, and proximal methods. We propose numerically stable reformulations and provide example implementations for each setting, validated by property-based tests.
Efficient sequential recommendation has emerged as a promising paradigm for providing real-time personalized service. To deploy models under resource-constrained environment, model compression becomes a necessity, yet existing methods typically apply uniform compression ratios, overlooking the heterogeneity in user intent complexity. This "one-size-fits-all" approach leads to a fundamental Capacity-Complexity Mismatch, where computational resources are either wasted on simple instances or insufficient for complex ones. To address this, we propose RACE (Ranking-uncertainty Aware Capacity-Adaptive Customization for Efficient Sequential Recommendation), a framework that dynamically adapts both model structures and input sequences with uncertainty-aware compression ratios. To ensure capacity-adaptive inference, RACE captures the model's intrinsic uncertainty of its predictions and transforms it into a proactive resource allocator. Specifically, we derive a list-wise ranking-uncertainty metric from the distributional stability of top-$K$ predictions, which serves as a measurable proxy for instance complexity. A lightweight predictor is then trained via self-supervision to anticipate this uncertainty from early-layer features. Guided by this signal, RACE performs instance-specific slimming in two dimensions: a hypernetwork-style channel sensitivity extractor identifies parameter redundancy, while importance-aware progressive sequence filtering mitigates data redundancy. Extensive experiments on four popular datasets demonstrate that RACE outperforms state-of-the-art baselines on accuracy, efficiency, and adaptability.
A unified framework is proposed that characterizes supervised deep classification as the decoding of error-correcting codes in the Shannon sense. Cross-entropy optimization is shown to align softmax classifiers with maximum-likelihood decoding, where logits map to Hamming distances to class-specific codewords. This correspondence explains the emergence of standard divergence objectives (e.g., Kullback-Leibler, Jeffreys) and provides a geometric basis for the stability of decision boundaries. Furthermore, the geometries of Neural Collapse and LeJEPA are identified as optimal constellations for decoding under isotropic noise. Unlike classical Error-Correcting Output Codes, deep networks are shown to learn effective codebooks in the feature space implicitly. This insight is leveraged to derive novel criteria for lossless quantization, improved knowledge distillation, and architectural regularization based on code redundancy. Finally, this perspective is extended to self-supervised encoders, offering a cohesive explanation for the geometry of latent representations in modern neural models.
Long-horizon LLM agents can coordinate tools and applications to complete complex, multi-step tasks. As their interaction histories grow, effective context management becomes essential for reliable and efficient execution. Existing methods reduce context through summarization, pruning, or history selection. However, shorter contexts do not necessarily support better decisions: summaries can omit exact details needed for later actions, while retaining entire interactions can carry substantial irrelevant information. We propose CarryOn, a lightweight context manager that pairs compact execution state with fine-grained, source-linked evidence. A learned selector and adaptive retention policy keep relevant evidence available without additional LLM calls for context management. We introduce two complementary evaluation metrics: $S_α$ prioritizes task success while accounting for resource use, and normalized cost per success (NCS) measures the resources spent per successful task. Across AppWorld, OfficeBench, and Tau2 Telecom with three LLM Actors, CarryOn achieves 75.7% macro-averaged task success, exceeding the strongest baseline by 6.7 percentage points. It also achieves the best average scores on both metrics.
Implicit neural video representations are notoriously slow to build, which does not cross the basic practical threshold for most industrial applications. Here, we present Q-GAS, a coordinate-queryable video representation that couples multi-resolution explicit 3D feature grids with a compact SwiGLU-style quadratic-gating (QG) decoder and an error-aware, prediction-free sampler. The explicit grids avoid hash-collision coefficient sharing at the cost of modestly higher storage yet greatly boost the expressive capacity; the QG decoder introduces low-rank local second-order interactions to improve the nonlinear feature response. During fitting, a global error map periodically refreshes a high-error pool, and each batch mixes pool samples with uniform samples, which is free from the auxiliary model to predict difficult samples, thereby further escalating the efficiency. On UVG Beauty/Bosphorus/Jockey, Q-GAS fits and quantizes under 50 s on one RTX PRO 6000, decodes at over 370 FPS with mixed precision and about 146 FPS in FP32. In our PyTorch implementation, Q-GAS trains and quantizes 5.5-7.2 times faster than the evaluated Instant-NGP baseline and 19-28 times faster than RNeRV. Under a 90 s budget, it improves PSNR/MS-SSIM/LPIPS over the evaluated NeRV-family pipelines at comparable BPP over the measured overlap. We have shared our code at https://github.com/MillenRosen/Q-GAS for readers' free use.
Frontier LLM agents that query databases with tools are among the most accurate text-to-SQL systems, but per-query calls to a hosted model are slow, costly, and expose sensitive organizational queries to external APIs. For example, a Claude Opus 5.5 agent serving 10,000 queries a day costs over $70,000 a year at list price. To what extent can local hardware already owned by an organization offload this burden without compromising the agent's accuracy? We present LuthorSQL, a non-autoregressive 7B text-to-SQL model that serves as a rapid System 1 gate for a slower LLM agent (System 2), escalating queries only when uncertain. LuthorSQL makes a single forward pass to predict slot scores over a bounded grammar for query plans, compiles and runs up to 32 candidate plans, and chooses an answer using a trained validator. On its own, LuthorSQL reaches 68.03% execution accuracy on BIRD's hidden test set at 0.84 s per question on a single GPU, maintains its accuracy in 9 GB with 8-bit quantized weights, and transfers zero-shot to Snowflake, BigQuery, MySQL, PostgreSQL, and DuckDB within 3.1 points of its native SQLite accuracy. Combined with Claude Opus 5.5 in a pre-registered confidence cascade over 227 questions on databases published after Opus's training cutoff, LuthorSQL answers nearly half of all questions locally and cuts API cost by 33.3% at the same accuracy. Finally, contamination probing shows that Opus reproduces 48-92% of public benchmark gold queries verbatim without seeing the schema; however, its tool-assisted accuracy holds on renamed schemas and reaches 95.6-97.4% on post-cutoff databases. Code and data: https://github.com/IAMIbrahimmemon/luthorsql
Knowledge distillation (KD) is a highly effective paradigm for transferring capabilities from large language models (LLMs) to smaller, more specialized student models. In commercial deployment where frontier models are restricted behind black-box APIs, sequence-level distillation remains a potent vector for competitors to clone proprietary capabilities at a fraction of the development cost. Existing defenses mostly rely on surrogate models to manipulate next-token sampling, which increases computational overhead and real-time inference latency. In this work, we propose a lightweight defense that dynamically injects style-steering vectors into the teacher model's activations upon malicious queries. This framework offers exceptional flexibility, allowing hosts to customize target defensive styles and dynamically blend them to maximize stylistic variance. Crucially, this inference-time intervention preserves semantic utility and fluency during malicious queries, misleading adversaries into believing that they are extracting pristine teacher outputs, while standard consumers continue to receive original, unmodified responses. Our experiments demonstrate that this framework successfully disrupts student learning, achieving substantial anti-distillation performance while incurring negligible overhead and preserving text naturalness. Our code is available at https://anonymous.4open.science/r/distill-resist-38C6/.
A developer running a code assistant on a laptop downloads one model file, and local inference moves the energy of every request onto that machine. A common rule says a larger model at lower precision tends to beat a smaller one at higher precision. We ran 24 GGUF files of Qwen2.5-Coder at four sizes on a fanless MacBook Air M2 and priced each in energy per solved task on 542 EvalPlus tasks. The result is a selection guide indexed by Ollama's listed file size, with an evidence label on each pick. At 1 and 2 GB it picks the q4_K_M files of the 1.5B and 3B models, both on the correctness and energy frontier under every accounting. At 3 GB as listed, the larger-model rule picks 7B-q2_K, the only measured 7B file that fits, and there it does not pay. That file brings no significant gain in solves over the 2 GB pick, 3B-q4_K_M, and the 90% CI limits any 3B-q4_K_M deficit to 2.4 points. Yet 7B-q2_K needs 2.03 times the energy per solved task on CPU, GPU and memory rails, or 1.66 to 1.81 times on whole-system power. On HumanEval+, five alternating session pairs give a ratio of 1.925 against 1.956 in the main grid. At 8 GB, 7B-q6_K buys a significant gain over the 2 GB pick at a higher energy price. From 1.5B up, q4_K_M trails the highest precision measured at its size by less than 3 points (90% CI). We release per-task results, power logs and the analysis script.
This paper presents a multi-GPU version and a multi-GPU recursive version of the parallel partition method for systems of linear algebraic equations with tridiagonal coefficient matrices and their implementations in CUDA. Computational experiments on one and two GPU cards, for different system sizes and with different number of recursive steps are conducted. The results are analyzed and discussed.
GPU communication libraries such as NCCL use Streaming Multiprocessors (SMs) for data movement and synchronization, slowing overlapped computation at a cost that grows with interconnect bandwidth. We present Zero-SM NCCL, which communicates within and across scale-up domains without launching any SM kernel. It synchronizes through CUDA stream memory operations, moves data asynchronously with copy engines within a scale-up domain, and drives network RDMA from a stream-ordered CPU proxy. It supports data-movement collectives such as AllGather and AlltoAll, plus one-sided ncclPutSignal and ncclWaitSignal APIs. On GB200 NVL72 and B200 NVL8 systems, Zero-SM collectives reach up to 1.36x higher AllGather bandwidth than NCCL's legacy SM-based kernels and cut the median slowdown of overlapped GEMMs from 33% to 4%, at a higher latency per operation. In Megatron-LM training, they accelerate FSDP by up to 3.5%, and one-sided puts accelerate pipeline parallelism by up to 15%. Zero-SM communication is not a universal win, however. Through an end-to-end study across models and parallelism strategies, we show that it pays off for large transfers in GPU-bound workloads and for deep pipelines with large bubbles, but loses on small messages and host-bound workloads. We distill these findings into guidelines for enabling zero-SM communication in NCCL and building it into other libraries.
Multi-view 3D transformers such as VGGT reconstruct a scene from a set of unposed images in a single forward pass, predicting camera poses, depth and 3D points, but the cost of their global attention grows quadratically with the num- ber of images. Accelerators borrowed from large language models assume that each token attends to only a few oth- ers; on these models they lose accuracy or gain little speed. We study the global attention of three multi-view transform- ers on seven datasets and find that every layer plays one of three roles: it averages over the whole scene, matches points across many images, or anchors on the first image, whose coordinates the outputs use. Building on this anal- ysis, we propose Regime-AWare ATTention Approximation (RAWATTA), a training-free method that runs attention in 8- bit precision and keeps 16-bit precision only in the layers where a worst-case calibration check requires it. Its fast set- ting, RAWATTA-Fast, also approximates attention in a few averaging and anchoring layers, while matching layers and the first image keep all their keys. On a 12 GB GPU at 50 im- ages, RAWATTA runs VGGT faster than the released code, mostly through our CUDA engine, and loses at most about one point of pose accuracy (AUC@3) on any dataset; RAWATTA-Fast is slightly faster and less accurate. Both fit 200 images, where the released code runs out of memory at 100.
We introduce Gamer, a generalist agent framework built for long-horizon tasks in complex interactive environments. Its end-to-end architecture combines perception, memory management, and multi-step reasoning. A shared training and deployment pipeline supports interaction with both conventional 2D graphical interfaces and complex 3D simulated worlds. We first continue pre-training on a dataset constructed from tens of thousands of hours of device-level keyboard and mouse interaction. These trajectories span simulated games and embodied environments, allowing the model to learn environmental dynamics across both physical and virtual domains. Sparse-thinking instruction tuning then substantially reduces redundant computation while preserving the reasoning needed for complex decisions. We train the policy for long-horizon interaction with a stable multi-turn reinforcement learning method that addresses finite context windows, accumulated rollout errors, and sparse rewards. For real-time interaction, a cloud-edge inference engine serves numerous simulated game environments concurrently, supports unbounded-context execution, and keeps online inference latency low. Across multiple challenging 3D open-world games, Gamer demonstrates solid generalization and operational efficiency comparable to skilled players. It also achieves competitive results on a range of conventional 2D game benchmarks. These results suggest that even repetitive keyboard and mouse trajectories can support substantial gains from scaling data and learning algorithms. Training on these trajectories helps Gamer learn complex skills more steadily in environments with both physical and digital properties. This approach offers a path toward generalist agents that can interact across digital and physical worlds.
Matrix multiplication is the dominant computational kernel in deep learning, yet its growing scale strains conventional von Neumann hardware in energy efficiency and bandwidth. Neuromorphic architectures offer a promising alternative, but mathematical equivalence between matrix multiplication and spiking computation does not guarantee direct execution on real neuromorphic hardware, as the specific hardware paradigm may still require additional data recovery, bit-weight reconstruction, or state readout after neuromorphic execution. This paper presents BPU-SpikeMM (Brain Processing Unit-based Spike Matrix Multiplication), an exact spatio-temporal spike encoding framework for matrix multiplication on real neuromorphic hardware. We prove the exact equivalence of spike-encoded matrix multiplication under the BPU execution paradigm, distinguish mathematical equivalence from native direct-readout equivalence on BPU hardware, and build a spatio-temporal offloading pipeline supporting multiple encoding schemes. Furthermore, we extend the framework to a multi-dimensional encoding design space and evaluate the applicability of different encodings on real BPU hardware in terms of correctness, temporal resource cost, spatial resource cost, and event-driven metrics. The results confirm native direct-readout equivalence across all evaluated encoding schemes and show that their applicability to BPU execution depends on joint trade-offs among temporal cost, spatial resource consumption, and event activity.
Training language models across low-bandwidth networks requires reducing synchronization overhead. Local optimization methods such as DiLoCo communicate only after several local steps, but outer gradient averaging and the outer update remain on the critical path. Existing computation-communication overlap methods avoid this stall through delayed or replica-local updates, which can degrade convergence or cause replicas to drift. We propose LACCO (Local Accumulation with Computation-Communication Overlap), a two-stage algorithm that overlaps global communication with local computation while applying globally aggregated updates shared by every replica, without introducing staleness in the outer update. We establish convergence guarantees for smooth non-convex objectives. Experiments on language-model pre-training with FineWeb show that LACCO consistently outperforms communication-overlapped baselines such as Eager and Delayed DiLoCo. These gains remain robust as model size scales up to 0.5B parameters and across a range of communication intervals and numbers of replicas up to 16 workers, while incurring only negligible-or fully overlappable-computational or memory overhead, which demonstrates the strength of LACCO.
We argue that the Transformer revolution is not adequately understood as a merely quantitative expansion of deep learning, but as the emergence of a new paradigm based on a different principle of inference. We interpret the Transformer as a system in which token representations are transformed by parameterized transformations, or transforming concepts. During inference, some of these transformations are dynamically constructed from the current input: previously acquired transformations are first adapted and then applied. At the mechanical-structural level, this dynamic processing is realized through output-weight interconnections, whereby outputs of some networks determine the weights of others. The resulting mode of processing, which we call Sequence-level Interactive Dynamic Parallel Processing (SIDPP), provides a framework for explaining strong prompt sensitivity and opens research directions concerning interpretability, control, alternative AI architectures, neuromorphic systems, and possible biological realizations of dynamic processing.
Modern multi-vendor GPU environments force a choice between portable code and native performance. GPRat is an open-source Python library for exact Gaussian process regression built on the HPX asynchronous many-task runtime, tiled algorithms, and CPU/CUDA BLAS backends. We contribute a UXL Foundation oneMath backend with a hybrid HPX/SYCL design, extending GPRat to AMD and Intel GPUs. The new SYCL backend comes within 3% of native CUDA on an NVIDIA A30 at the largest problem size and outperforms the GPflow and GPyTorch libraries by up to 1.75x and 2.17x on the A30 and up to 1.79x and 4.62x on an AMD Instinct MI210. On an Intel Arc B580, a consumer-class card with FP64 throughput below our CPUs, the unmodified SYCL backend trails GPRat's CPU backend. Overall, oneMath's dispatch to vendor-native BLAS lets a single SYCL codebase match CUDA on NVIDIA and outperform established Python GP frameworks on AMD, without vendor-specific code.
Processing-in-memory (PIM) has been proposed as an alternative computing paradigm for training Deep Neural Networks, including Graph Neural Networks (GNNs). Despite these advancements, training GNN workloads on PIM devices necessitates off-chip memory access. This off-chip access is expensive in terms of performance and energy, thereby impacting the overall energy efficiency of the training process on PIM platforms. In this paper, we propose a novel hybrid training framework called ERGo that automatically switches from a full-parameter phase to a parameter-efficient phase during the training process with negligible loss in predictive accuracy. The parameter-efficient phase employs low-rank representations of GNN weights, effectively reducing the number of trainable parameters. This helps in reducing the off-chip access during the end-to-end training process on PIM architectures, leading to notable improvements in both performance and energy efficiency. ERGo outperforms the conventional full-parameter GNN training on a PIM-based platform by up to 3.15x in speedup and enhances energy efficiency by 10.5x. Furthermore, ERGo offers additional advantages when utilized on heterogeneous architectures incorporating non-volatile memory (NVM) and static random-access memory (SRAM). Training on the heterogeneous NVM-SRAM architecture typically utilizes NVM for the forward-pass and SRAM for the backward-pass. However, NVM cells suffer from low device endurance and repeated write operations due to weight updates can lead to poor lifetime. ERGo improves the lifetime of NVM devices by freezing the weights and preventing weight updates in the parameter-efficient phase. Experimental results demonstrate that ERGo improves the lifetime of heterogeneous NVM-SRAM architecture by up to 33x in comparison to full-parameter implementation.
Earthquakes remain among the most destructive natural hazards, necessitating reliable monitoring and early warning systems for effective risk mitigation. Recent advances in machine learning (ML) and deep learning (DL) have significantly improved seismic signal analysis, enabling more accurate event detection, phase picking, classification, and magnitude estimation. This study presents a systematic review of ML- and DL-based approaches for earthquake monitoring, with particular emphasis on Distributed Acoustic Sensing (DAS) as an emerging technology for high-resolution, real-time seismic observation. Following the PRISMA 2020 guidelines, a systematic literature search was conducted across Scopus, Web of Science, IEEE Xplore, and Google Scholar, yielding 252,223 initial records. After applying the predefined publication period, removing duplicate records, conducting relevance screening, and performing eligibility assessment, 138 peer-reviewed studies published between 2021 and 2025 were retained for detailed analysis and synthesis. The review reveals a significant transition from conventional signal-processing techniques to advanced artificial intelligence-based approaches, including Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM) networks, Bidirectional Long Short-Term Memory (BiLSTM) networks, Transformer-based architectures, hybrid models, and Bayesian learning methods for uncertainty quantification. The findings further demonstrate that Distributed Acoustic Sensing (DAS) has emerged as a transformative sensing technology because of its dense spatial coverage, high spatial resolution, and continuous monitoring capability. However, several challenges remain, including the lack of standardized datasets, limited model generalization across diverse geological settings, insufficient model interpretability, high computational complexity, and the limited integration of uncertainty-aware approaches for real-time seismic monitoring. This review identifies these critical research gaps and highlights promising future research directions, including multimodal data fusion, interpretable artificial intelligence, physics-informed learning, self-supervised learning, and robust uncertainty quantification for next-generation intelligent seismic monitoring systems. Unlike previous review studies that primarily focus on individual machine learning techniques or conventional seismic monitoring, this review provides a comprehensive and systematic synthesis of recent advances in machine learning, deep learning, and Distributed Acoustic Sensing (DAS), identifies current research gaps, and offers practical recommendations to guide future research on intelligent earthquake monitoring systems.
Many digital systems courses teach on FPGA boards whose vendor tool is no longer maintained: the Digilent Basys2 and Nexys2 (Xilinx Spartan-3E) and Nexys3 and Atlys (Spartan-6) need Xilinx ISE 14.7, discontinued since 2013 and hard to install on the computers and tablets students use today. The author presents Silinx, a free (AGPL-3.0) HDL design platform for teaching that keeps ISE's Project Navigator workflow but runs in a web browser with nothing to install. A module can be written in VHDL or Verilog, in C or C++ through Silinx HLS, which is compatible with AMD's Vitis HLS and passes all 73 of its introductory examples, or drawn as a schematic, an ASM chart, a state diagram, a Boolean function with its Karnaugh map, or a Petri net, from which its HDL is generated. Designs are checked with explanations written for beginners, simulated, run on an emulated board, built, and programmed onto a real board over WebUSB. The implementation flow is open and runs in the browser: Yosys and nextpnr compiled to WebAssembly, with Silinx's own device data, timing model and bitstream writer for the Spartan-3E and Spartan-6, built by black-box observation of ISE's documented tools; Lattice iCE40 boards use the existing open iCE40 tools, and further families are planned. ISE is the oracle: Silinx's bitstreams are byte-identical to those ISE writes for the same placed and routed design, and pass ISE's design rule check. Against ISE at the same optimization goal, the open flow uses 15% fewer slices with 14% shorter critical paths when optimizing for area, and 21% fewer slices with 8% shorter critical paths when optimizing for speed (geometric means over 45 open-source designs built by both flows). The author also describes how the platform was developed with AI coding agents under test-first project rules.
High-Level Synthesis (HLS) provides a practical route for mapping graph algorithms to FPGA accelerators, but the benefit of loop-level pipelining depends strongly on the global structure of the target algorithm. This paper presents an implementation-oriented hardware characterization of Breadth-First Search (BFS), Dijkstra’s single-source shortest-path algorithm, and Bellman–Ford on a Xilinx Alveo U250 using Xilinx Vitis HLS and compressed sparse row (CSR) graph storage. The critical edge-processing loops of all three kernels achieve an initiation interval of one. For the reported hardware validation cases, BFS processes a 1.4-million-edge graph in approximately 1.45 million cycles at a 10.4-ns reported clock period, while Dijkstra and Bellman–Ford require approximately 5.2 million and 10.4 million cycles, respectively, at 12 ns on 100-thousand-edge validation cases. Normalizing latency by the input edge count gives approximately 1.04, 52, and 104 cycles per input edge for BFS, Dijkstra, and Bellman–Ford, highlighting that identical edge-loop II does not imply identical algorithm-level efficiency. The difference arises from algorithm structure: BFS is dominated by frontier-driven streaming, Dijkstra introduces repeated minimum-vertex selection, and Bellman–Ford performs repeated whole-graph relaxation rounds. Resource measurements further characterize the hardware cost of the three mappings. The study focuses on FPGA kernel behavior and HLS mapping rather than competitive CPU–FPGA speedup, providing design insights for future graph accelerators implemented with modern HLS flows.
This work presents the design, RTL implementation, and functional verification of SACU-DMA, a Secure Access Control Unit for Direct Memory Access (DMA) with a fully compliant ARM AMBA AXI4 master interface. Through a single-cycle combinational security validation stage that evaluates the entire byte ranges of both source and destination addresses against a programmable protected region, the controller enforces hardware-level memory protection, structurally eliminating the Time-Of-Check-Time-Of-Use (TOCTOU) vulnerabilities of software-based protection schemes. Four key architectural features are integrated into a single synthesizable Verilog HDL module: (1) an eight-state Finite State Machine orchestrating the full AXI4 read-thenwrite pipeline across all five channels (2) hardware-enforced region protection with four simultaneous boundary comparators (3) a synthesis-safe combinational clk_en activity indicator that flags channel activity for use by higher-level SoC power management—no ICG cell, latch, or gated-clock path is instantiated in this revision, so every flip-flop remains on the ungated system clock and (4) a three-source independently maskable interrupt subsystem with sticky pending flags and software-clearable latches, plus multi-beat transfer support with automatic 8-byte address incrementing. A System Verilog testbench with an AXI4 slave model and a transaction scoreboard executed seven directed test cases covering all functional paths, achieving a 100% pass rate (23/23 checks). For single-beat 64-bit transactions, the controller provides approximately 114 MB/s throughput at 100 MHz. Vivado Power Analysis reports 0.705 W total on-chip power (0.624 W dynamic, 0.081 W static) at 26.3 °C junction temperature. The design targets security-critical SoC integration in trusted execution environments, automotive safety systems, and IoT edge platforms.
Conventional RTL-based ALU designs tend to suffer from long carry propagation delays, excessive partial-product overhead, and significant dynamic power dissipation during idle cycles, limiting their suitability for high-performance and power-aware digital systems. In response, the proposed architecture integrates three co-optimised novelties to address these limitations. Dedicated Brent–Kung parallel prefix adder and subtractor blocks, which bring the carry propagation delay from O(N) to O(logN), improving computational throughput. A Radix-8 Booth encoding multiplier reduces the number of partial products from N to N/3, significantly lowering multiplication latency and area. Most distinctively, a latch-based Integrated Clock Gating (ICG) cell is incorporated to suppress clock toggling when the ALU is idle, thereby reducing dynamic switching power consumption by an estimated 35–40% during inactive cycles. The ALU supports arithmetic, logical, and shift operations and generates standard processor status flags including zero, carry, overflow, and negative. Unlike prior work that treats fast arithmetic and clock gating as independent optimizations, this work quantifies how a reduced-latency Brent–Kung datapath directly increases the exploitable idle window for latch-based clock gating, and evaluates this coupled effect on a Xilinx Virtex7 FPGA using Verilog HDL.Post-implementation results show a Brent–Kung critical-path depth of 6 logic levels versus 8 for a ripple-carry adder. The 8-bit ALU uses 186 LUTs and 21 registers, with the ICG adding only 1 LUT and 1 register ( 0.5% overhead). Functional verification confirmed correct operation across all opcodes, while switching analysis estimated a 35–40% reduction in dynamic power under a 50% idle duty cycle.
This paper proposes a 16-tap FIR filter structure for FPGA realization based on Radix-4 Booth multipliers and approximating multipliers’ output depending on the runtime control signal. This structure has four modes of operation: exact multiplication, 2-bit truncation, 4-bit truncation, and 6-bit truncation. Truncating selected least significant bits of the multiplier outputs provides runtime-configurable accuracy through different levels of output approximation. The FIR filter has been developed in Verilog HDL and checked on ModelSim simulation for signed input samples and fixed symmetric coefficients. Mean absolute error for approximate modes is equal to 5.75, 63.00, and 348.50 for 2-bit, 4-bit, and 6-bit truncation, respectively. Synthesis has been carried out using Xilinx ISE 14.7 with Artix-7 FPGA device family. The result of synthesis uses 1216 look-up table (LUT) cells, 145 flip-flop elements, 28 input/output (IOB) cells, and no DSP48E1 cells. Estimated minimum clock period equals 7.837 ns, which corresponds to estimated maximum clock frequency of 127.601 MHz. The results demonstrate that the proposed architecture is capable of achieving runtime configurable accuracy through the use of general-purpose FPGA resources.
The majority of arithmetic operations are based on binary addition and hence depend on the speed of binary adders; therefore, the efficiency of ALU can be judged on the basis of adder performance. Traditional RCA adders are simple to design, but their propagation delay increases with the increase in number of carry propagation levels. CLA and CSA adders reduce the propagation delay by parallel generation of carry. In order to enhance the throughput of the design, the paper discusses the implementation of a 32-bit four stage pipelined CLA adder along with an ALU and compares its performance with traditional RCA, CLA and CSA. These designs have been implemented using Verilog HDL and synthesized using Xilinx Spartan-7 FPGA using Vivado 2025.2. The architecture consists of four stages synchronized with the clock; these stages include PG generation, lower 16-bit and upper 16-bit carry computation and sum/flags generation. A delay of 6.703 ns and a frequency of around 149.2 MHz have been achieved. Hardware verification of design using Xilinx Integrated Logic Analyzer confirmed correct pipelined operation and accurate ALU status flag generation, thereby demonstrating its suitability for high-throughput FPGA-based VLSI systems.
In this paper, we present an advanced architecture that can enable us to implement a two-user power domain non-orthogonal multiple access (PD-NOMA) system by employing Field Programmable Gate Arrays (FPGAs). Unlike the inherent constraints associated with conventional orthogonal multiple access (OMA) systems, NOMA exploits the advantages of power-domain multiplexing to significantly increase spectrum efficiency without using any extra bandwidth. This single-precision floating-point synthesizable Verilog model is developed from scratch which covers various components such as the transmitter design which includes a LFSR imperfection generator, Rayleigh fading, addition of AWGN noise, and finally a successive interference cancellation (SIC) component. Performance analysis is conducted by measuring the bit error rate (BER) and sum-rate for the NOMA system with two users which are located at a random distance from the base station. The system is implemented in a Xilinx Artix-7 XC7A35T FPGA with a total consumption of 0.155 W and a negative time slack is observed for the worst-case scenario of 0.868 ns at a speed of 100 MHz. Moreover, the sum data rate improvements in the range of 30–40% are achieved in the NOMA system compared to the existing OMA case for all transmit power levels.
In modern exceptional mobile communication networks, NOMA (Non-Orthogonal Multiple Access) has become an established technology that enables users to be served at the same time without requiring new frequencies to be assigned. In this article a large scale downlink NOMA system based on beamforming and the use of clustering is developed. The users which are in the same direction are formed as cluster. In this work, we consider two users in each cluster. Both the near and far users within the same cluster are able to use similar time and frequency resources in the power domain multiplexing method. At the receiver end, the near user’s signal can be decoded from the combined signal by employing successive interference cancellation (SIC) technique. Digital beamforming based on zero forcing (ZF) is used to minimize inter-cluster interference. Unlike prior simulation-only studies, this work presents an ASIC-oriented RTL implementation and complete physical-design evaluation. The VLSI design flow were utilized to review, synthesis, test and analyze the designed downlink NOMA beamforming RF front-end circuitry before finalizing the design for evaluation in both timing, area, and power categories. The obtained results demonstrate that the proposed clustered NOMA beamforming approach achieves successful RTL-to-GDSII implementation with timing closure at 100 MHz and low hardware overhead, making it suitable for practical real-time wireless communication systems.
In this paper, the design and validation of a SystemVerilog-based 1×3 packet router is presented through a holistic approach of RTL design and verification based on assertions combined with post-synthesis analysis for Network-on-Chip (NoC) applications. The proposed router is based on a modular architecture that includes three independent FIFO buffers, a synchronizer, an FSM controller, and a register module to implement deterministic packet routing, buffering, flow control and parity handling. Functional verification is performed using a layered SystemVerilog-based testbench with transaction-level components and scoreboard-based checking. SystemVerilog Assertions (SVA) are used to verify protocol sequencing, FIFO safety, synchronization logic, parity generation, and FSM state transitions. The results of the simulations verify the accuracy of the routing, congestion control, and parity checks under typical operating conditions. The verified RTL is synthesized using Cadence Genus Synthesis Solution (45nm technology), with a total silicon area of 9256.16 µm2, positive worst-case setup slack of 5.322 ns, and total power consumption of 757.1 µW. The proposed methodology offers a unified approach for the implementation, verification and hardware realization of reliable network-on-a-chip (NoC) packet routers.
This paper presents a synthesis-level comparative study of four 16-bit binary adder architectures: Ripple Carry Adder (RCA), Carry Look-Ahead Adder (CLA), Brent–Kung Adder (BKA), and Kogge–Stone Adder (KSA). The architectures were described in Verilog HDL, functionally simulated using Cadence SimVision, and synthesized using Cadence Genus. Four timing targets of 10 ns, 8 ns, 6 ns, and 5 ns were applied while keeping the corresponding RTL architecture unchanged. The reported implementation metrics are area, total power, propagation delay, and timing slack. The power reports additionally separate leakage, internal, and switching components. For the evaluated stimulus frame, the measured total powers are 78.2400, 70.4407, 79.2930, and 89.3166 µW for RCA, CLA, BKA, and KSA, respectively. The corresponding leakage contributions are 3.89%, 4.25%, 6.39%, and 7.47%, while switching contributes 65.73%, 72.97%, 66.25%, and 62.49%. The measured delays are 4580 ps, 3852 ps, 2102 ps, and 1556 ps, respectively. Because the same RTL implementation is synthesized for each timing target, the reported area, power, and propagation delay remain unchanged for an architecture; the timing slack changes with the timing budget. The work therefore interprets the four-constraint experiment as a timing-margin sensitivity study rather than as evidence of constraint-driven physical optimization. The study is intentionally limited to synthesis-level results; larger word lengths and post-layout parasitic effects are identified as important extensions rather than being inferred from the 16-bit data.
Cross-reality systems connect physical and virtual participants through heterogeneous interfaces; however, supporting communication and collaboration across these interfaces remains a design challenge. We report an in-the-wild study (N = 45) of a public installation linking a gesture-controlled gallery wall to an immersive virtual gallery through synchronized content and a digital mirror. Questionnaires showed no significant differences in user experience or presence. Observations and interviews suggest that verbal communication and a reliable cross-reality view supported coordination.
Throwing is a vigorous, ballistic gesture that is awkward to stage in Virtual Reality (VR): a bare controller is weightless, offers no resistance to load against, and can slip out of the hand mid-throw. We present JavRail, a VR javelin-throwing demo built around a rail-guided physical proxy. A handle rides on an inclined overhead rail; a compression spring gives the draw-back a rising load, and a stop at the end of the rail receives the forward thrust and catches the carriage after the user releases the handle, preventing it from flying freely. Carriage motion and rail angle drive a virtual spear in a short wave-defence game. The demo leans on a motor pattern visitors already own rather than teaching a new one, and brings a gesture that normally needs a field into a booth-sized footprint. We position JavRail as an arcade-style exertion demo rather than a training instrument, and report its mechanical design, re-arming behaviour, and safe operating procedure.
As physical distance separates users, XR enables new forms of remote communication and shared experiences. Current systems rarely reflect everyday experiences of visiting someone in their personal space, as AR leaves each user in their own environment, while VR places users in a potentially unfamiliar virtual space. Both lack either a shared spatial frame of reference or the contextual cues of personally meaningful surroundings, which are important factors for social interactions. We investigated how sharing environmental context affects social presence in remote XR collaboration. In an in-situ experiment (N = 30), dyads of friends collaborated remotely from their homes across four conditions: both using AR in their own spaces, both using VR to share an unfamiliar environment, a hosting condition in which one participant remained in their real environment in AR while their partner virtually visited a reconstruction of that space in VR, and a visiting condition representing the opposite perspective. Social presence was measured via the Networked Minds Social Presence Inventory, the Inclusion of Other in the Self scale, behavioral indicators, and post-experiment interviews. Results show that sharing environmental context influences social presence, with the asymmetric hosting condition eliciting higher social presence and co-presence, compared to the conventional VR setting. Interview findings highlight the more natural and familiar sense of virtual visits. Our work provides empirical evidence that combining a personally meaningful physical environment with a remote VR visitor can support higher social presence in remote XR collaboration. These insights inform the design of XR systems that support remote interactions resembling in-person visits, offering novel opportunities for social connectedness across distance.
In VR, hand tracking—which uses cameras and image processing embedded in HMDs to recognize the user’s hands as direct input without a physical controller—has become widely adopted as an intuitive interaction paradigm; however, precise repositioning of densely packed small objects remains a significant challenge. Because fingers have a finite width, they can simultaneously occlude multiple adjacent targets in dense configurations, making it difficult to determine which object the user intends to manipulate. In this work, we focus on the thinness of virtual hand fingers. Displaying slimmer fingers is expected to reduce situations in which a single finger simultaneously overlaps multiple targets, thereby clarifying the intended object. To investigate how finger thinness affects precise target repositioning in VR, we conducted two experiments. Although fingers can technically be made arbitrarily thin, excessive thinning risks degrading the sense of body ownership and disrupting natural, intuitive interaction in VR. In Experiment 1, we varied finger thinness across seven levels to identify the range of finger thinness over which body ownership is maintained. In Experiment 2, using a pick-and-place task, we examined the effects of finger thinness on completion time, error rate, pinch distance, the number of erroneous grasps per trial, and subjective usability ratings. Our results show that finger thinness had a significant main effect on all objective measures: compared to the baseline (× 1.00), the thinnest condition (× 0.10) reduced completion time by 1.1 s (4.68 → 3.57 s), error rate by 12.2 percentage points (\(46.4\% \rightarrow 34.2\%\)), and pinch distance by 1.2 mm (6.86 → 5.65 mm), and reduced the number of erroneous grasps per trial from 0.84 to 0.52. Subjective usability ratings were also affected by finger thinness, with comfort and accuracy peaking at an intermediate level (× 0.40) rather than at the thinnest condition. This work quantitatively characterizes how finger thinness affects both body ownership and precise target repositioning, providing new design insights for virtual hands in VR.
Gesture-based locomotion techniques are widely used in virtual reality, with arm-based gestures serving as a commonly used locomotion metaphor. This study systematically investigates three seated arm-based gesture modalities defined by their primary joints: the shoulder (Upper-arm Swinging; US), elbow (Forearm Swinging; FS), and wrist (Hand Swinging; HS). Through a three-part experiment, we evaluated the sense of presence and subjective workload, expected walking speeds, and physical tempo limits. Quantitative measures showed no significant differences in the sense of presence across arm‑based conditions, but they revealed a difference in physical demands, with US > HS. Furthermore, we found that the expected walking speed for US was significantly higher than for FS and HS. We also identified that the maximum achievable input tempo follows the order HS > FS > US. Overall, our analysis revealed an inverse relationship where heavier gestures with lower maximum input tempos correlate with faster expected walking speeds. Similarly, we found a positive correlation between a gesture’s moment of inertia and the user’s expected walking speed, suggesting that these physical constraints may act as sensory cues related to virtual propulsion scaling. These results point to a design trade-off in which the gesture conveying the strongest sense of forward motion (US) is also the most physically demanding and the hardest to sustain.
The safety and success of an interactive product strongly depend on early usability testing. However, creating prototypes is time-consuming and costly. Virtual reality (VR) prototypes offer a promising approach, but virtual results must be validated for transferability. To address this need for validation, we used a gesture-based medical interface as an exemplary test case to investigate the applicability of the VR methodology. We conducted an experiment with 41 participants using a 2 × 2 mixed design: two environments (between-subject: physical or VR-based prototype) and two interface variants (within-subject: one with and one without deliberately placed usability issues). Environment differences were largely non-significant, and post-hoc equivalence tests further excluded a large discrepancy between VR and physical testing for the number of usability issues. A large discrepancy in relative interface effects was also excluded for task completion time and error count, though the latter was confounded by a floor effect. The SUS showed no significant environment effect, but equivalence could not be established at any threshold, leaving this result inconclusive. However, gesture count did significantly differ between environments, with no equivalence established. Nevertheless, VR-specific hardware and control dynamics should be considered when interpreting these findings. This work provides insights into the validity and comparability of metrics in early-stage VR usability tests for non-haptic products.
Spatial interaction between users and virtual characters displayed in real space enhances the presence of the characters and strengthens the connection between them. In this study, we focus on aerial imaging technology that forms display images in real space, aiming to realize a system in which virtual characters presented as aerial images can move three-dimensionally within the physical environment. Previous studies on real aerial images using a retrotransmissive element have proposed optical systems in which the depth position of the aerial image is controlled by changing the angles of both the display and the optical element. However, the conventional optical configuration suffers from a limited vertical field of view, requiring users to align their eye height with a predefined viewpoint. To address this issue, this paper formulates the design parameters for an angle-controlled depth-movable aerial imaging method with the objective of expanding the field of view, and proposes an optimization method for these parameters. The proposed approach enables the design of system parameters that maximize the depth movement range of the aerial image while satisfying the field-of-view requirements for unclipped observation by users with different eye heights. Furthermore, assuming the use of a 420 mm square glass retrotransmissive element, we performed parameter design based on the proposed method and implemented a prototype system for evaluation. Optical display performance evaluation demonstrated that the prototype achieved a larger depth movement range than the conventional linear movement method, with no clear depth-dependent degradation in image quality when using the glass MMAP. Additionally, we observed that, in the aerial image depth-moving method based on angle control, the changes in luminance and image quality accompanying depth variation differed between the glass and resin retrotransmissive elements used in this study.
Virtual reality (VR) lets users embody non-humanoid avatars, but quadrupedal locomotion is challenging because its gait differs fundamentally from human bipedal movement. Prior work shows that gesture congruence and physical posture each enhance embodiment, yet their influence on users’ self-perception during quadrupedal locomotion remains underexplored. We drive a quadruped avatar’s plantigrade gait from hand movements alone, avoiding whole-body fatigue. Because the forelimbs map onto human arms, the technique splits into two axes, body posture and hand-gesture mapping. In a within-subjects study (N = 24) crossing two postures (pseudo-quadrupedal, standing) with two gesture mappings (Rowing, Swinging), both manipulations enhance body ownership and agency, with no detectable interaction. Avatar-consistent self-perception further differs across the four configurations, reflecting posture, viewpoint, and scale rather than locomotion alone. Finally, perceived skill is higher for Rowing in the pseudo-quadrupedal posture than in standing, which also raises physical workload. These results unify the two frameworks into a single design space for plantigrade quadrupedal avatar locomotion that trades off embodiment quality against workload.
As remote work extends into everyday shared spaces, Extended Reality (XR) offers new potential for remote collaboration beyond screen-based tools, supporting co-presence and embodied interaction. Prior work has examined XR use in public settings and how users adapt to physical and social constraints. Less is known about what happens when remote XR collaboration, which adds remote colleagues, shared objects, and professional expectations, enters everyday places such as cafés and coworking spaces, and how the two reshape each other. Through 12 speculative design workshops with 24 designers and researchers across six everyday spaces, we find participants treated XR collaboration not as a digital layer added to the physical world, but as a temporary professional workspace with its own professional logic, embodied demands, and colleague expectations. We show how this XR workspace conflicts with everyday spaces across three dimensions: whether XR work is recognized as legitimate (governance), who can perceive what is happening (social), and how much room XR work requires (physical). These dimensions are interconnected: for example, large gestures in a park can trigger social judgment, which raises questions about whether such work belongs there. We therefore reframe XR collaboration as a socio-spatial coordination challenge between XR workspaces and everyday spaces, and propose design directions for making their requirements, constraints, and boundaries mutually visible and negotiable.
Deploying Transformer-based models on power-constrained edge devices is increasingly limited by the energy cost of data movement between memory and compute units. The compute-in-memory (CIM) architecture is a promising approach for energy-efficient Transformer inference by performing matrix operations where weights are stored. Nevertheless, Transformer layers interleave GEMM-dominated kernels with non-GEMM operators such as Softmax and LayerNorm, whose analog-to-digital conversion (ADC) and control overheads offset the in-memory computation gains. This paper proposes OPAL, a cross-layer co-design framework that jointly schedules precision, sparsity, and operator-level execution within a unified CIM pipeline. OPAL matches precision to per-operator sensitivity, couples block-sparse attention with reconfigurable subarray gating, introduces a mixed-signal LayerNorm path to reduce ADC invocations by 75%, and overlaps heterogeneous operators through a conversion-aware pipelined dataflow. Evaluation on BERT-Base demonstrates that OPAL achieves 0.66× latency and 0.64× energy compared with the SOTA baseline while attaining 1.22× energy efficiency, and maintains over 88% F1 on SQuAD.
Early and reliable detection of neonatal seizures is crucial for timely intervention to prevent long-term neurological damage. Continuous EEG monitoring remains challenging due to the need for expert interpretation and real-time processing constraints. Although deep learning-based approaches have shown promising detection performance, their deployment in hardware-constrained clinical wearable environments is limited by high computational complexity and a lack of interpretability. This work presents a hardware-efficient implementation of the convolutional neural network (CNN)-based seizure detection framework for neonatal EEG using the Helsinki University Hospital dataset. To support real-time operation, a custom hardware architecture is developed, comprising a RISC-V-based control processor tightly coupled with a dedicated CNN accelerator. The accelerator consists of configurable processing element (PE) arrays along with the control unit for efficient dataflow management, enabling parallelized convolution operations with reduced memory overhead. The proposed hardware architecture was implemented on a Zynq-7000 FPGA and evaluated on neonatal EEG data, achieving 83.87% sensitivity, 93.29% specificity, and 91.66% accuracy.
For real-time face authentication on edge devices, there is a need for lightweight deep learning models and energy-efficient hardware architectures to achieve high recognition accuracy in low latency and low power consumption. In this paper, the design and FPGA implementation of a MobileNetV3-Small AI accelerator for real-time face authentication on the Xilinx PYNQ-Z2 platform is presented, which is energy-efficient. The proposed framework utilizes MobileNetV3-Small with ArcFace loss for discriminative facial embedding generation and adopts post-training INT8 quantization, which saves memory and computation cost while maintaining the recognition performance. The custom accelerator is synthesized in Verilog-2001 and consists of INT8 multiply-accumulate units, processing elements, an 8 ×8 systolic array, on-chip BRAM-based feature and weight memories, dense-layer controller, vector normalization and embedding normalization modules. To minimize unnecessary switching activity and reduce dynamic power consumption during inference, a clock-enable based clock gating technique is used. Evaluation of the training process was performed experimentally and resulted in an accuracy of 93.70%, precision of 95.87%, recall of 91.33%, F1 score of 93.55%, and ROC-AUC of 98.09% after convergence. The strategy for a low-power FPGA implementation is proposed, which allows for low-latency inference and efficient use of resources, making it suitable for real-time biometric authentication on an edge device.
To reconstruct the super-resolution volumetric medical images like CT, MRI we have introduced a FPGA accelerated edge intelligence framework. We propose a ligh weight 3D multi-task architecture called EfficientVolSeg-Recon Net with 4.8 million parameters. This framework features deep supervision, depthwise separable convolutions and squeeze-excitation attention. For the present study, we have customized a systolic-array FPGA accelerator over Xilinx Zynq Ultrascale+. This setup achieves 24.1 FPS at INT8 precision. Also, it only consumed 8.7 W power. An evidential deep learning based orchestration engine is developed to handle the variable network conditions and clinical urgency. This engine can dynamically decides the edge and cloud offloading to achieve a mean dice similarity coefficient to 0.912, with sub 50-ms latency on an average. We also demonstrated that it achieves about 4.8 times energy efficiency on GPU utilization when compared with the other GPU edge device platforms. We have validated our proposed setup in a hospital to cloud 5G based platform testbeds. Even under the fluctuating bandwidth of 10-1000 Mbps, it shows robust performance.
Discovering symmetries from observational data provides interpretable insights into dynamical systems and priors for scientific machine learning. However, existing symmetry discovery methods face two limitations. Many methods represent symmetries implicitly, making it difficult to obtain explicit infinitesimal generators that can serve as physical priors for learning dynamical systems. Meanwhile, methods that explicitly identify generators often rely on predefined function libraries, limiting their ability to capture complex symmetries involving high-order polynomials or transcendental functions. To address these limitations, we propose LieDiscover, a framework that explicitly identifies infinitesimal generators through an iterative symbolic search process that dynamically expands the symbolic function library to capture complex functional forms. Specifically, at each search step, we design an encoder-decoder architecture to generate a candidate symbolic expression. An adaptive singular value decomposition (SVD)-based evaluation is introduced to assess how each candidate contributes to the recoverable symmetry subspace. The resulting evaluations are incorporated into step-wise rewards to guide reinforcement learning toward discovering richer symmetry structures with concise symbolic representations. Experiments on eight partial differential equation (PDE) benchmarks demonstrate that LieDiscover recovers explicit generators involving higher-order polynomial, exponential, and trigonometric terms, while achieving more accurate symmetry subspace recovery and improved robustness to noise. Furthermore, the recovered generators enable effective symmetry-based data augmentation, improving the accuracy of downstream neural PDE solvers.
Background: The application of Deep Learning (DL) in Minimally Invasive Gynecological Surgery (MIGS) has accelerated rapidly, driven by the need for enhanced intra-operative navigation, surgical phase recognition, and predictive analytics. The efficacy of these applications is fundamentally determined by the underlying neural network architectures, yet a comprehensive technical synthesis of their comparative performance and clinical utility in the gynecological context is absent from the literature. Objective: To provide a comprehensive scoping review of the DL architectures deployed in MIGS, evaluating their technical mechanisms, clinical applications, performance metrics, and limitations. Methods: A scoping review was conducted following PRISMA-ScR guidelines, drawing from a systematic search of 55 included studies identified across five major databases. Studies explicitly detailing neural network architectures applied to gynecological surgical tasks were analyzed, with a focus on Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM) networks, Deep Neural Networks (DNNs), Transformer architectures, and hybrid models. Results: CNNs remain the foundational architecture for spatial feature extraction in surgical video analysis, achieving accuracy rates of 85–97% in anatomical landmark identification and instrument detection. LSTM networks, frequently integrated with CNNs in hybrid architectures, significantly improve surgical phase recognition by capturing temporal dependencies across video sequences. Transformer architectures, including Vision Transformers (ViTs), have demonstrated superior performance in complex multi-modal tasks, outperforming CNN-LSTM models in surgical workflow understanding. Emerging applications include augmented reality navigation, automated endometriosis lesion recognition, and AI-driven prediction of perioperative outcomes. Conclusion: The DL landscape in MIGS is evolving from static image analysis toward dynamic, spatio-temporal surgical understanding. While CNNs and LSTMs constitute the current standard, Transformers represent the emerging frontier. Critical priorities for future development include model interpretability, real-time edge deployment, and standardized benchmarking across diverse gynecological datasets.
Discovering optimal graph neural network (GNN) architectures for various tasks is both labor-intensive and time-consuming. To reduce human effort, graph neural architecture search (GNAS) has recently been utilized to automatically identify effective GNN architectures for specific tasks, achieving competitive or even superior performance compared to manually designed architectures. However, existing GNAS methods fail to identify optimal architectures in the presence of structural and semantic noise, where structural noise refers to missing or redundant edges within the graph structure, and semantic noise denotes inaccurate node representations derived from ambiguous node features such as textual vagueness or semantic ambiguity. In this paper, we address this problem for the first time via theoretical analyses and empirical evaluations. We discover that existing differentiable GNAS methods typically select architectures based on task-relevant information hidden in the graph, being highly sensitive to structural and semantic noise, which results in suboptimal selection of GNN architectures under noise. To handle the structural and semantic noise, we propose Curriculum-GraphLLM, a novel graphLLM framework for joint optimization of architectures, structures, and texts for denoised graph neural architecture search. The core idea is to jointly optimize GNN architectures, graph structures, and textual semantics as a unified denoising process during architecture search. Specifically, we first develop a dynamic topology updating mechanism to adaptively adjust the graph structure. Then, we jointly optimize the GNN architecture and graph structure through a curriculum-based iterative updating approach. To further deal with semantic noise on text-attributed graphs (TAGs), we introduce LLMs as an auxiliary text modeling module to refine textual semantics and guide the co-optimization of text representations, graph structures, and GNN architectures. We conduct extensive experiments to show that our proposed Curriculum-GraphLLM achieves consistently competitive or superior performance compared with existing baselines, especially under structural and semantic noise.
Early detection of breast cancer is essential for reducing mortality, but deploying large deep learning models in resource-constrained clinical settings remains challenging. This study evaluates knowledge distillation (KD) from a ResNet-50 teacher to a MobileNetV3-Small student for classifying breast ultrasound images as benign, malignant, or normal, compared with the same student trained without KD. We partitioned the Breast Ultrasound Images Dataset (BUSI), containing 780 images from 600 patients, using a fixed, stratified 70/15/15 image-level split. Unavailable patient identifiers prevented patient-level splitting, constituting a study limitation. We applied Total Variation Chambolle denoising before training. Grid search selected temperature T = 2 and weighting coefficient α = 0.5, achieving 88.89% validation accuracy. Across five random seeds, the KD student achieved 82.05% ±1.60% test accuracy and 80.93% ±2.04% macro F1-score, compared with 82.74% ±1.64% and 81.88% ±2.17% for the no-KD student, and 83.59% ±4.25% and 82.41% ±5.62% for the teacher. Equivalence testing supported KD–teacher equivalence within a ±5-percentage-point margin (p=0.0498). Paired Wilcoxon tests found no evidence of improvement over the no-KD baseline (p=0.625) or denoising-free training (p=0.4375). The student had 15.5 times fewer parameters and approximately 2.5 times faster inference than the teacher. These findings support compact models for breast ultrasound classification but provide no evidence that KD outperforms simple fine-tuning of the same lightweight architecture on this dataset.
Neural Architecture Search (NAS) has so far rarely been applied to Mixture-of-Experts (MoE) models, and existing MoE designs leave the alignment between experts and the structure of the data to emerge on its own. We propose an architecture search framework that makes this alignment an explicit search variable: the assignment of data clusters to experts is optimised jointly with the per-expert architectures. We cast the joint problem as a cluster-aware likelihood maximisation, show that it coincides with the incomplete-data maximum likelihood of a latent-variable mixture, and solve it by a generalised Expectation-Maximisation procedure whose otherwise intractable expert-quality term is supplied by an adaptively refined surrogate. We prove that the iterates converge whenever the surrogate errors are summable, and that at every limit point no candidate the search produces improves the true objective. On a heterogeneous image-classification mixture the method recovers the underlying domain partition on 95% of clusters without ever observing domain labels, and on that benchmark and a four-domain time-series forecasting one alike it outperforms the MoE and NAS baselines that likewise use no label information.
This paper introduces NepScript Genesis, a Neural Architecture Search (NAS) framework for automated Generative Adversarial Network (GAN) discovery, applied to conditional Devanagari handwritten digit synthesis. We compare five NAS strategies against a carefully constructed Deep Convolutional GAN (DCGAN) baseline (FID=332.28). Architecture selection utilizes a two-stage pipeline guided by a novel domain-aware evaluation metric (Enhanced Score). Results demonstrate that Adaptive Exploration achieves the optimal quality-efficiency trade-off, attaining an FID of 79.12 -- a 76.19% improvement over the baseline -- and the highest mode coverage among the NAS strategies (Recall=0.531) in under one GPU-hour. Furthermore, we demonstrate that incorporating script-specific structural heuristics into the search phase prevents early-stage mode collapse. In a downstream low-resource evaluation, augmenting 250 real training samples per class with GAN-generated digits from the best NAS model improves CNN classification accuracy from 91.0% to 96.5% (+5.5 percentage points), demonstrating that NAS-optimized synthesis produces digits of sufficient quality to benefit practical recognition pipelines when real data is scarce.
The proliferation of Internet of Things (IoT) devices and the demand for real-time processing have positioned edge computing as critical infrastructure for deploying deep learning models near the data source. Hardware constraints, including limited memory, narrow computational budgets, and strict power limits, challenge deployment of large-parameter neural architectures. This review examines hybrid architectures integrating convolutional neural networks (CNN) with attention mechanisms for local feature extraction on resource-constrained edge devices. Following the PRISMA 2020 protocol, a multi-stage search was conducted exclusively on Scopus, yielding 3,571 records, screened until 121 high-quality studies were included. The review addresses five research questions covering architectural trends, efficiency strategies, performance trade-offs, application domains, and federated learning for privacy-preserving deployment. Findings show that hybrid CNN-Attention architectures outperform pure CNN and Transformer baselines, with 3.2% average accuracy improvement while maintaining competitive inference latency on platforms such as NVIDIA Jetson and Raspberry Pi. Depthwise separable convolution, efficient channel attention, and token aggregation emerged as dominant compression strategies, with health imaging and fault diagnosis as leading domains. The review concludes that hybrid architectures represent the state of the art for edge-oriented local feature extraction, with future directions toward heterogeneous node generalization and multi-modal sensor fusion for distributed IoT networks.
JIT compilers are core components of modern language virtual machines, and their bugs can crash the VM or silently miscompile programs. Modern JIT compilers perform multiple rounds of optimizations on graph-based intermediate representation (IR), where each round modifies, transforms, or eliminates IR nodes (referred to as node evolution). Bugs arise when the combination of multiple evolution paths violates inherent constraints of the IR. However, existing fuzzing tools lack fine-grained awareness of IR-level node evolution and struggle to construct the compound IR structures required to trigger such evolution, limiting their ability to explore the evolution combination space effectively. We propose EvoFuzz, an IR-aware fuzzing framework for Java virtual machine (JVM) JIT compilers, comprising three components: Node Mutators, which induce IR nodes of different semantic categories through source-level mutations and are portable across compilers; Evolution Profiling, which tracks evolution at the optimization-path granularity and defines a set distance metric as the feedback signal for fuzzing; and Combination-Aware Scheduling, which learns sequential dependencies among mutators via Markov dependencies to optimize mutator selection. We implement EvoFuzz as a practical fuzzing system for production JVMs and evaluate it on multiple versions of OpenJDK HotSpot, OpenJ9, and GraalVM. EvoFuzz discovered 45 bugs, of which 37 have been confirmed and 10 have been fixed by developers. Compared with state-of-the-art JVM fuzzers, EvoFuzz improves code coverage in the HotSpot C2 compiler module by approximately 7%, triggers 1.7–1.9 × as many unique optimization paths on average, and achieves 3.0–3.5 × the baselines’ median set distance at 12 hours. These results show that node-evolution-aware feedback is an effective signal for exposing correctness bugs in production JIT compilers.
Sanitizers such as ASan, UBSan, and MSan are widely used to detect memory-related errors and undefined behavior. However, their high runtime overhead makes continuous use difficult in testing and production-like executions. A major source of this overhead is redundant sanitizer instrumentation: compiler-inserted checks often validate the same variable repeatedly along the same control-flow paths. Reducing sanitizer checks requires reasoning about the property each check validates and the program state on which it depends. We explore a conservative policy that eliminates only covered READ checks and retains WRITE checks and those whose semantics remain unresolved. This paper presents DeSan, an LLVM IR-level framework that reduces redundant sanitizer checks while preserving sanitizer semantics. DeSan identifies high-impact core checks and recovers the checked variable for each check through backward IR tracing. It then classifies checks as READ, WRITE, or UNKNOWN and builds per-variable check graphs to find covered READ checks. DeSan preserves all WRITE and UNKNOWN checks. When safe, it removes redundant READ checks along with their sanitizer-only IR slices, including shadow computation, comparisons, branches, and isolated report paths. We evaluate DeSan on SPEC CPU2006, SPEC CPU2017, and 20 open-source projects. Across 163 valid sanitizer configurations, core checks cover a median of 96.62% of inserted checks, and DeSan removes a median of 51.48% of all checks while preserving evaluated bug-detection capability. Compared with ASAN− − for ASan and SanRazor for UBSan, DeSan achieves competitive or stronger overhead reduction.
Flash floods in hilly and mountainous regions develop rapidly, which makes localized risk detection and timely warning difficult because of sparse ground-sensor coverage, challenges in integrating different data sources, and communication delays in remote areas. This paper presents a Flash Flood Prediction System for Hilly Regions using Multi-Source Data. The system combines rainfall information, Sentinel-1 satellite imagery, IoT ground-sensor telemetry, and historical and terrain data. Sensor data is filtered, validated, and preprocessed at the edge before integration. A hybrid artificial intelligence approach uses Random Forest to analyse terrain and historical information and Long Short-Term Memory (LSTM) to identify trends in live data, and a late-fusion process combines the outputs into localized risk levels: Low, Watch, Warning, and Evacuate. A Flask-based machine-learning backend serves predictions through an API, a React-based dashboard displays real-time risk on a map, and alerts are designed to reach communities and response authorities through SMS, application notifications, and local sirens. An IoT sensor-node prototype and a live monitoring interface are presented. Quantitative validation is identified as future work.
The growth of the Internet of Things (IoT) exposes resource-constrained devices to an expanding range of cyber attacks. Machine learning-based intrusion detection systems (IDSs) can detect these attacks, but they spend the same inference budget on every network flow, even when a flow is easy to classify. We propose LIMITS (Lightweight IoT Mutual-Information Two-layer System), a hybrid IDS in which a depth-limited Decision Tree resolves every flow that reaches a leaf of high training purity and escalates the remaining flows to an XGBoost classifier. The routing threshold is the mean leaf purity of the trained tree, so it needs no tuning, and mutual-information ranking with a parsimony rule selects the features. An evaluation over five stratified splits of two well-known IoT benchmark datasets showed that LIMITS resolved about half of the traffic in the first layer, nearly doubling the inference speed of the standalone ensemble with comparable detection performance and unchanged false-positive and missed-attack rates, that the operating point could be adjusted without retraining, and that LIMITS reached a higher mean macro-F1 than two re-implemented recurrent detectors on both datasets (significantly, only on CICIoT2023), with much shorter fitting and inference times. LIMITS does not, however, reduce the false alarms of XGBoost on benign traffic, which remain the main obstacle to deployment.
Buildings are a key application domain for improving energy efficiency and sustainability, while Industry 5.0 emphasizes sustainable and human-centric digital transformation. This study presents a comparative framework for IoT-based residential energy consumption forecasting, combining time-series models with explainable artificial intelligence (XAI)-based feature analysis. Using a publicly available dataset from a residential building in Leicester, United Kingdom, the study predicts the next-hour total power consumption of six monitored appliances using a 24-h historical window and measurements available at or before the forecast time. Ridge Regression, conventional machine learning models, iTransformer, and TimeFilter are evaluated using 993 out-of-sample observations. SHapley Additive exPlanations (SHAP) and Local Interpretable Model-Agnostic Explanations (LIME) support feature analysis and selection. With all features, Ridge Regression achieves an RMSE of 35.869 W, an MAE of 23.261 W, and an R2 of 0.265. Feature selection has varying effects across models and evaluation metrics and does not necessarily improve predictive performance. Persistence and 24-h seasonal-naïve methods serve as baselines. The findings assess explainable IoT-based forecasting without requiring a full Digital Twin infrastructure and may offer insights relevant to Industry 5.0-oriented building management. Further validation across buildings and operating conditions is needed to assess generalizability.
Generative AI (GenAI) is redefining the technological architecture of enterprise digital transformation, yet its domain structure and convergence topology remain uncharacterized at the patent level. This study applies a three-stage patent analytics pipeline- Cooperative Patent Classification domain mapping, keyword-rule topic assignment across 15 GenAI application archetypes, and weighted co-occurrence network analysis- to 9,692 Lens.org patent records from ten jurisdictions, of which 7,623 published from 2018 onward form the analytical corpus. Filings grew at a compound annual rate of 42% over 2018–2025. Domain mapping identifies Computing & Information Systems (69.5%), AI & Machine Learning (36.5%) and Business & Commerce (25.2%) as a three-layer infrastructure, model and application stack. Large language model development and retrieval-augmented generation together account for 64.5% of filings, while agentic AI and edge inference clusters remain under 1% each. The co-occurrence network (26 nodes, 151 edges, density 0.465) identifies Computing & Information Systems as the dominant convergence hub (composite score 0.759) and Healthcare Informatics as a structural broker (betweenness 0.138) disproportionate to its filing volume. Cross-domain patenting peaked at 2.34 domains per patent in 2021 and moderated to 1.65 by 2025. The findings extend general-purpose technology convergence theory to GenAI and provide a structural map for technology managers.
The rapid expansion of the Internet of Things (IoT) has increased the volume of data generated at the network edge, where many devices operate under severe memory, processing, and energy constraints. In this context, Tiny Machine Learning (TinyML) enables machine learning (ML) tasks to be executed on microcontrollers, smart sensors, and other resource-constrained platforms. However, the literature remains heterogeneous regarding the algorithms evaluated, energy-measurement methods, reported metrics, influencing factors, and optimization strategies. This study presents a systematic mapping of energy-aware TinyML research. It identifies the main models and learning approaches, classifies energy-evaluation methodologies and metrics, examines the technical factors affecting energy consumption, and organizes the reported optimization techniques into a structured taxonomy. The findings highlight the frequent use of convolutional and lightweight architectures, direct measurements and analytical estimations, and factors related to model architecture, hardware, communication, numerical precision, and input characteristics. The optimization strategies were grouped into model compression, hardware-aware design, system and communication optimization, and data and processing optimization. The results also reveal limited standardization in experimental platforms, measurement procedures, and reporting practices, limiting reproducibility and direct comparison across studies. This mapping provides an updated overview of the field and guidance for developing and evaluating more energy-efficient TinyML solutions.
Water entering a distribution system may be lost physically through leaks, lost economically through metering and billing issues, or used without being invoiced, making Non-Revenue Water (NRW) a recurring concern for metropolitan water utilities. In order to identify anomalous network behavior, locate leaks, and assist with operational decisions, machine learning (ML), the Internet of Things (IoT), advanced metering infrastructure (AMI), hydraulic modeling, and cloud/edge computing are rapidly being integrated. With a focus on leakage detection, localization, prediction, and decision support, this paper summarizes the development of data-driven NRW management. Data sources, pre-processing and feature engineering, algorithm families, validation procedures, and operational deployment are the five dimensions around which the evidence is arranged in the review, which follows an organized systematic-review approach based on PRISMA 2020. In addition to hydraulic-model residuals and network architecture, pressure, flow, acoustic, and AMI data are analyzed. While CNN, LSTM, and graph-based models are increasingly employed for spatial, temporal, and topological patterns, traditional machine learning models like support vector machines, random forests, and gradient-boosting techniques are still helpful when datasets are moderate and manufactured features are instructive. Moving from high-performing laboratory or simulated trials to resilient utility-scale deployment with sensor sparsity, noise, demand unpredictability, network changes, and limited labeled leak occurrences is another important research problem, according to recent studies. Thus, a closed-loop NRW management system that connects sensing, data quality control, machine learning inference, hydraulic validation, field verification, repair priority, and model retraining is proposed in the review. Standardized datasets, leakage-specific assessment methods, cross-network validation, explainable and physics-informed machine learning, cybersecurity, model-drift management, and economic decision support are important research needs. The study comes to the conclusion that machine learning (ML) should not be used as a stand-alone substitute for hydraulic engineering and field practice, but rather as a decision-support layer inside a larger water-loss-control program.
IoT-Driven AI and Mathematical Optimization for Sustainable Energy Management in Mechanical Systems explores the integration of the Internet of Things (IoT), artificial intelligence, and mathematical optimization techniques for improving energy efficiency and sustainability in mechanical systems. The book covers intelligent energy monitoring, predictive maintenance, data-driven decision-making, energy consumption forecasting, system optimization, and sustainable resource management. It highlights how real-time sensor data, machine learning algorithms, and mathematical models can be combined to optimize system performance, reduce energy consumption, minimize operational costs, and support environmentally responsible engineering practices. This book serves as a valuable resource for students, researchers, academicians, and engineering professionals working in intelligent manufacturing, mechanical engineering, energy systems, and sustainable technologies.
With the rapid urbanization of modern cities, public transport demand has significantly increased, leading to higher energy consumption and accelerated equipment degradation, affecting system efficiency and sustainability. In fact, urban rail networks generally rely on static models and centralized systems unable to deal with real-time fluctuations. This paper proposes a lightweight digital twin (DT) framework for urban rail transit systems, integrating artificial intelligence (AI), the Internet of Things (IoT), and blockchain technology to enable real-time energy optimization and predictive maintenance. IoT sensors continuously collect operational and energy data across the network. Machine learning models, including Random Forests and Neural Networks, dynamically predict energy demand, while a novel genetic algorithm (GA) optimizes train synchronization to maximize regenerative braking energy recovery. Moreover, a blockchain layer is integrated to ensure the security of critical energy transactions and maintenance events. The approach is validated through DT simulations using 2.7 million operational records from the Paris metropolitan rail (Metro) systems (Régie Autonome des Transports Parisiens/Paris Transport Authority, RATP). Simulation results showed an increase in regenerative energy utilization by 92.4%, a reduction in overall peak-hour energy consumption by 12.7%, and an improvement in the early detection of mechanical anomalies by 24% compared to traditional baseline models. The integration of a lightweight DT combining AI, IoT, and blockchain has the potential to provide a more scalable, more resilient, and overall, more economically effective system for urban rail networks. By offering an adaptive, data-driven digital representation, the proposed framework supports sustainable transportation initiatives and can be generalized to other global urban railway networks.
Serverless computing is widely adopted for its simplicity, autoscaling, and fine-grained billing. However, GPU-based serverless frameworks often inherit CPU-oriented designs, leading to inefficiencies in machine learning (ML) inference. Specifically, the keep-warm policy, effective on CPUs, is suboptimal on GPUs due to rigid hardware binding, and replica-based autoscaling incurs significant memory overhead and frequent model swaps, which are amplified by large machine learning models. To address these problems, we propose HybridFaaS, a hybrid autoscaling strategy that combines pipelined and tensor-parallel scaling to save memory with replica-based scaling for flexibility, reducing model swapping overhead without undue scheduling complexity. Evaluations demonstrate that HybridFaaS significantly outperforms state-of-the-art systems with 80-97% higher throughput and 99-147 × lower P99 tail latency.
Online LLM serving is increasingly constrained by GPU memory capacity during decode. In length-skewed workloads, a small fraction of long requests keeps large key-value (KV) caches resident across many decode steps, exhausting GPU high-bandwidth memory (HBM), throttling admission, and reducing goodput for the short-request majority. At the same time, GPU servers provision substantial CPU and DRAM resources, and recent Intel CPUs add Advanced Matrix Extensions (AMX) for matrix-heavy kernels. The challenge is to use these host resources without placing CPU work on the per-token GPU critical path. We present Chorus, a GPU–CPU cooperative serving system that uses AMX-equipped CPUs as a second decode lane for long requests with persistent KV state. Chorus keeps prefill and short-request decode on GPUs, offloads selected long-request decode once to distributed AMX CPU groups, and releases the corresponding GPU KV pages so the GPU lane can admit more short requests. To satisfy online service level objectives (SLOs), Chorus combines pressure-aware offload admission, tensor-parallel AMX decode that aggregates CPU compute and memory bandwidth, and AMX kernels specialized for small, mixed-length decode batches. Across popular LLMs and production-derived workloads, Chorus achieves the highest SLO-attaining goodput among baselines, improving peak goodput over SGLang by up to 1.26 ×. At the same offered load, Chorus delivers 1.3–11.4 × SGLang’s goodput when GPU-only serving enters time-to-first-token (TTFT) collapse, and improves over CPU-offload baselines by up to 4.3 ×.
Training generative models on long contexts is increasingly bottlenecked by attention activation GPU residency: activations must be on GPU when they are produced and later reused, yet keeping all of them resident limits context length. This paper presents HeadFlow, a runtime system that recasts attention activation management as a head-group lifetime planning problem. HeadFlow exploits a long-context asymmetry: attention compute grows rapidly with sequence length, while each head group’s activation state grows much more slowly, creating windows that can hide movement. It names semantic activation slots, places them across CPU pinned memory, direct GPU–SSD storage, staged GPU–CPU–SSD paths, or recomputation, and executes the resulting plan with asynchronous save, staged prefetch, and load. We implement HeadFlow by extending FlashAttention-compatible operators with plan-aware asynchronous activation movement, and evaluate it across long-context language and multimodal generative training. On text workloads, HeadFlow trains with 512K–1M-token contexts. Its throughput is 1.33 × –1.67 × that of state-of-the-art sequence- and context-parallel baselines. To our knowledge, HeadFlow is the first system to train a 24B model with full parameters at 1M context on only eight 96GB H20 GPUs. On image and video workloads, the same runtime extends the maximum trainable effective tokens by 4.0 × and achieves a 1.32 × speedup. Loss and attention-gradient checks confirm that these gains come from exact activation management rather than approximate attention or lossy execution.
GPU Confidential Computing (CC) enables secure LLM inference in production, combining a CPU TEE with NVIDIA’s GPU TEE to protect models and user data from a malicious cloud operator. As long-running LLMs are failure-prone, traditional cloud systems rely on backups, and live migration using checkpoint and restore to deal with failures. In particular, during LLM serving, GPUs build up a massive key-value (KV) cache that accelerates the decode process. But when using GPU CC, these recovery methods fail, as the GPU’s memory state, such as the KV cache, can’t be efficiently checkpointed. Checkpointing GPU memory requires encrypting the GPU data and transmitting to the CPU over the PCIe, which the CPU then decrypts using its shared key with the GPU. This encryption and decryption sequence over the PCIe leads to a 5.5 × bandwidth penalty. The decrypted data is then re-encrypted with a new set of keys shared by the CPU and a destination server which adds overhead to the already-long checkpoint latency. LLM serving is stalled for the entire duration of checkpointing; a secure full-dump of an 80 GB H100 memory takes roughly 11 s. Alternatively, tracking write operations and selectively checkpointing these written pages is a preferable path, as only a small fraction of the KV cache is updated after each decode round. Current GPUs, however, lack the ability to track dirty pages. We present FLASH, a checkpoint/recovery (C/R) system for GPU CC that is both secure (providing confidentiality, integrity, and freshness against a malicious operator) and low latency. FLASH eliminates needless decryption of GPU state during checkpoint creation. FLASH preserves the GPU-encrypted HBM data as is without decryption. The CPU then creates a heterogeneous encryption package: (1) the CPU’s own memory state and the key used to encrypt the GPU CC is encrypted into a single bundle using keys shared by the CPU and a destination server that stores the checkpoint. (2) The GPU-encrypted HBM data is preserved as a second bundle within this heterogeneous package. This approach eliminates redundant decrypt-then-reencrypt overheads for GPU CC state. FLASH also presents a novel approach to track dirty pages to enable incremental checkpointing. FLASH enhances the GPU’s unified virtual memory (UVM) driver to downgrade any GPU page that is already checkpointed to a read-only page. It then uses the write fault mechanism to track modified pages at a 2 MB granularity. This approach requires no source code modifications to the LLM code base. Lastly, FLASH classifies never-written allocations, such as model weights, and removes them from repeated checkpoint copies while continuing to track them for correctness. On an H100 with Intel TDX running vLLM, FLASH reduces checkpoint stall latency up to 9.4 × over state-of-the-art GPU C/R systems.
Disaggregated Large Language Model (LLM) inference has gained popularity by decoupling the compute-intensive prefill stage from the memory-intensive decode stage, enabling the use of low-end, compute-focused GPUs for prefill and high-end, memory-rich GPUs for decode. This separation reduces cost while maintaining high throughput. However, transmitting Key-Value (KV) data between the two stages can induce high overhead, especially for long prompts. Additionally, the computation time overhead for prefill and decode is key for optimizing Job Completion Time (JCT), and KV data size can become prohibitive for long prompts and sequences. Existing KV quantization methods alleviate transmission and memory overhead by storing KV in low-bit formats, but attention still runs on dequantized BF16 KV in every iteration, incurring substantial recurring dequantization overhead and offering no reduction in attention compute. We propose Homomorphic Acceleration via Compression of the Key-Value cache (HACK) for disaggregated LLM inference. HACK keeps KV in a highly compressed INT2 form for transmission and caching, and it computes attention directly on the quantized KV. A lightweight approximation in the epilogue replaces per-iteration dequantization. HACK thereby reduces KV transmission, decode-time memory access, and attention computation together. Extensive trace-driven experiments show that HACK reduces average JCT by up to 70.9% compared to the disaggregated LLM inference baseline and by up to 53.5% compared to state-of-the-art KV quantization methods.
Multi-precision inference balances serving efficiency and model quality for large MoE LLMs under limited memory. However, changing expert precision at runtime exposes a fundamental tradeoff in existing systems: they either waste GPU memory by storing multiple precision replicas, or incur high I/O overhead by reloading whole expert tensors. Thus, achieving both dynamic precision and efficient execution remains an open challenge. We present MoE-Lego, an end-to-end MoE inference system that uses bitplane representation as the substrate for dynamic multi-precision execution. MoE-Lego first constructs a single nested bitplane artifact with routing-aligned calibration, so all supported precisions share the same base planes. Its runtime scheduler changes precision by moving, retaining, and demoting only the required bitplanes, making the I/O cost approximately proportional to the bit-width delta instead of the full expert size. Its GPU executor treats target precision as an execution dimension: it builds precision buckets for routed FFN work and computes each bucket with grouped bitplane kernels, avoiding fragmented per-expert computation. Across Qwen1.5-MoE, Qwen3-MoE, and Phi-3.5-MoE on A100 and RTX 4090, MoE-Lego achieves geometric-mean speedups of 3.72 × for prefill and 2.83 × for decode over the baselines, while preserving near-16-bit model quality. An additional evaluation on Jetson Orin demonstrates its applicability to SSD-backed expert offloading.
Live virtual machine (VM) migration enables more efficient data center operation by relocating VMs across hosts without service interruption. Because many cloud applications are both memory-intensive and interactive, however, live VM migration has proven difficult to implement with acceptable performance. The emergence of Compute Express Link (CXL) offers a new lever: memory pools shared across multiple machines. We present FM2, a new CXL-based live VM migration framework that makes migration faster and less disruptive. Before migration is needed, FM2 proactively stores a copy of VM memory in a CXL memory pool, substantially reducing migration time and accelerating resource reclamation. FM2 also efficiently promotes performance-critical data from the CXL memory pool to the destination’s local memory, enabling rapid post-migration performance recovery. By decoupling live VM migration from memory transfer, FM2 reduces interference with application traffic while further shortening migration time. Across a range of memory-intensive workloads, FM2 reduces total migration time by up to 68 × compared with QEMU’s hybrid migration across network and CXL interconnects, while keeping VM performance overheads below 2% and downtime under 150 ms.
Stateful Large Language Model (LLM) applications heavily rely on prefix caching to eliminate redundant prefill computations. To support massive concurrency, modern serving engines deploy hierarchical, multi-tier caches that offload less-active prefixes from limited GPU memory to spacious host DRAM. However, existing multi-worker request schedulers typically route traffic based solely on the longest prefix match, ignoring the highly heterogeneous data transfer costs between cache tiers. This oversight causes severe cluster-level load imbalance and triggers frequent GPU pipeline stalls as workers wait for slow host-to-device data transfers. To resolve these inefficiencies, we propose CacheSched, a scheduling system tailored for multi-worker LLM serving with hierarchical prefix caching. CacheSched is driven by a lightweight analytic performance model that estimates batch execution and data transfer times. Building on this model, CacheSched introduces two key mechanisms: (1) a heterogeneity-adaptive routing algorithm that dispatches requests based on predicted prefill costs, successfully balancing cache tier locality with cluster-wide worker load, and (2) a dynamic batch-matching algorithm at the worker level, which restricts host-memory prefix loads to the predicted compute window of the active batch, reducing GPU pipeline stalls. Evaluations demonstrate that CacheSched increases overall serving throughput by up to 2.58 × compared to state-of-the-art systems while satisfying strict Time-to-First-Token (TTFT) Service-Level Objectives (SLOs).
Long-context LLM inference is bottlenecked by GPU high-bandwidth memory (HBM), which the key–value (KV) cache rapidly exhausts. Existing designs compress, select, or offload KV entries, exposing trade-offs among HBM use, approximation error, and data movement. Our key insight is that locating the few tokens that dominate attention needs far less precision than computing the output over them: a heavily quantized cache can no longer reproduce the output, yet it still ranks tokens reliably. JetFire exploits this asymmetry with an approximate-then-correct (ATC) attention backend over hybrid memory. A dense low-precision pipeline scans a compact replica of the whole sequence in GPU HBM, fitting long contexts without dropping tokens. A sparse high-precision pipeline on the CPU corrects the dominant tokens from the full-precision replica in host DRAM. The correction overlaps the GPU value phase when transfer and computation fit within it. JetFire remains near-lossless relative to full attention, and JetFire-K3V2 achieves 1.3–3.0 × speedup in end-to-end execution time over TurboQuant within vLLM at the largest common contexts. Against offloading-based systems, it achieves 1.4–2.7 × speedup over NEO and 13–180 × over InfiniGen. JetFire is open-sourced at https://github.com/Xtra-Computing/JetFire.
The quantum switch applies two operations in a coherent superposition of their causal orders and is a simple example of indefinite causal order. No quantum circuit that uses each operation once in a definite order can even describe it, yet photonic experiments have demonstrated its physical realization. We ask how many calls to the black-box operations a quantum computer in the circuit model needs to reproduce the switch. If one of the black boxes is known to be unitary, three calls suffice. For general channels on a $d$-dimensional system, we prove three results. In all three, circuits may use classical feed-forward and unlimited ancillary quantum memory. First, no finite number of calls suffices for exact simulation in the worst case. Second, approximation to any fixed error below $1/2$, measured in half the diamond norm, requires $\Omega(d^{1/4})$ calls, exponential in the number of qubits. Third, an ordinary circuit can require exponentially many calls to obtain a single bit that the switch extracts with a few uses. We consider a device whose actual noise is dephasing in a fixed unknown basis and test it against a theoretical depolarizing model. The two channels have the same entanglement and average gate fidelities, so these figures alone do not distinguish them. At fixed noise strength, the quantum switch tells them apart with a number of device uses independent of $d$, whereas the circuit model requires $\Omega(d^{1/16})$ calls. Noise characterization is needed for quantum error correction, and this gives a provable query advantage of the switch in a task relevant to practical quantum computing. We invite future work that extends such advantages to more practical computing tasks.