Supplementary Dataset: Categorized Lexical Traps and Semantic Search Benchmark Logs

This repository contains the supplementary materials and experimental data supporting the research article: "A Comparative Analysis of Performance and Semantic Accuracy in Text Embedding Architectures via C# and ONNX". The dataset consists of two primary components: 1. Lexical Trap Evaluation Corpus (File: lexical_traps_dataset.csv) This file contains the custom dataset of 25 "Lexical Traps" used to evaluate the semantic robustness of the embedding architectures natively in .NET. The test cases span four major enterprise domains (IT Support, E-commerce, Finance, and HR). Each entry consists of a Query, a true Semantic Match, and a Lexical Trap—a deceptive document mathematically constructed to maximize Jaccard similarity (token overlap) while possessing a negated or inverted meaning. They serve as the strict evaluation space for measuring zero-shot semantic comprehension. 2. Experimental Benchmarks (File: benchmark_results_raw.txt) This section contains the raw C# execution logs and comprehensive per-query cosine similarity metrics for nine distinct open-source models (including FP32, INT8 quantized, and prompt-instructed variants). The logs rigorously document CPU inference latencies and semantic margins, supporting four critical analytical findings: Focus A: The Power of Asymmetric Prompting (e5-small-v2) Focus: Semantic Disambiguation & Prompt Instruction. Key Finding: Verifies that prompt-instructed models significantly outperform standard symmetric bi-encoders in avoiding negated traps. The e5-small-v2 architecture emerged as the optimal solution, successfully avoiding 48% of the lexical traps (12/25) and maintaining a strictly positive average semantic margin (+0.0061) at a highly efficient latency of 10.72 ms. Focus B: The Efficacy of INT8 Quantization (MiniLM-L6) Focus: Edge Computing & CPU Inference Optimization. Key Finding: Demonstrates that 8-bit integer quantization is highly viable for local .NET deployments. The INT8 variant reduced CPU inference latency by 36% (from 5.24 ms to 3.36 ms) compared to its FP32 baseline, while experiencing entirely negligible degradation in semantic accuracy (average margin dropped by only 0.0033). Focus C: Diminishing Returns in Heavy Architectures (MPNet-Base & BGE-Base) Focus: Parameter Count vs. Computational Cost. Key Finding: Illustrates the severe hardware penalties of scaling model size for local CPU inference. While heavy architectures like all-mpnet-base-v2 matched the highest accuracy (12/25 traps avoided), they incurred a nearly 200% latency penalty (averaging 31.24 ms), proving them inefficient for real-time enterprise micro-queries. Focus D: Context Optimization Penalties (Nomic-embed-v1.5) Focus: Long-context vs. Short-sentence Degradation. Key Finding: Highlights a critical architectural mismatch. Models optimized for massive context windows (up to 8192 tokens) performed poorly on short-sentence lexical traps. nomic-embed-text-v1.5 exhibited the highest latency overall (44.08 ms) and failed 88% of the queries, recording the worst semantic margin (-0.1554).