Computer Vision

References

A synthesis of selected conference talks and technical references. Citations link to the source material; they do not imply that every talk on this subject is included.

1. Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography

   Fischler and Bolles's 1981 RANSAC paper addressed geometric fitting when observations include gross mistakes, such as incorrectly identified image features. It repeatedly fits a model to a small random sample, finds observations agreeing within a tolerance, and refits promising models using their supporting observations. This separates finding a mutually consistent geometric explanation from merely minimizing error across every supplied match. The paper applies the approach to determining camera location from landmarks.

2. Foundations of Computer Vision: Stereo Vision

   Stereo estimates depth by matching scene points across spatially separated cameras and triangulating their viewing rays. For calibrated, rectified parallel cameras with baseline B and focal length f in pixels, disparity d=x_left-x_right gives Z=fB/d under the corresponding coordinate convention. First-order propagation gives depth uncertainty approximately σ_Z=(fB/d²)σ_d, so small disparities make distant depth sensitive to matching error. Additional viewpoints can reveal previously occluded surfaces and add geometric constraints only when usable correspondences exist. Low texture, specular reflections, noise, and occlusion make those correspondences uncertain.

3. Torchvision Bounding-Box Geometry Kernels

   Use continuous XYXY edge coordinates, x rightward and y downward, with box area (x2-x1)(y2-y1). For crop origin (left,top), resize factors sx=Wresized/Wcrop and sy=Hresized/Hcrop, then left/top padding (px,py), each corner becomes x'=sx(x-left)+px, y'=sy(y-top)+py. Inversion is x=(x'-px)/sx+left and y=(y'-py)/sy+top. These equations compose the implementation's crop subtraction, resize multiplication, and padding addition. Record actual resized dimensions because integer rounding can change the effective factors. Normalized coordinates must first be converted using the correct canvas size.

4. Foundations of Computer Vision: Optical Flow Estimation

   Optical flow estimates an H×W×2 field of apparent image displacement between frames. Under brightness constancy and a small-motion linearization, I_x u+I_y v+I_t=0. This is one equation for two displacement components: an isolated edge constrains motion normal to the edge but leaves tangential motion ambiguous, the aperture problem. Neighboring observations and temporal information add constraints under additional assumptions. Brightness motion need not equal physical scene motion: changing illumination, transparency, textureless surfaces, and occlusion can violate the model.

5. MVTec AD — A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection

   MVTec AD separates learning from defect-free examples from testing on normal and anomalous images. Its defects include scratches, dents, contamination and structural changes, with pixel-level reference regions. The paper distinguishes deciding whether an image contains an anomaly from locating subtle deviations in small regions. It evaluates both learned approaches and classical computer-vision methods, supplying a concrete inspection example where image-level decisions and defect localization require different evidence.

6. CS231n: Object Detection and Image Segmentation

   Classification produces image-level class scores without spatial extent. Classification plus localization adds a box for a designated object. Detection produces a collection of class-scored boxes for multiple objects. Semantic segmentation assigns a class to each pixel but does not distinguish separate objects of the same class. Instance segmentation produces separate object masks with class predictions. Useful representations are K class scores, N×4 boxes with N scores/classes, H×W semantic labels, or N×H×W instance masks. These describe predicted categories and image regions; deciding whether to alert, reject, or act is a separate application policy.

7. MAttNet: Modular Attention Network for Referring Expression Comprehension

   Referring-expression comprehension localizes the image region described by a natural-language expression. MAttNet scores candidates using subject appearance, image location and relationships to surrounding objects. Its location representation includes normalized box coordinates and relative area; its relationship module combines neighboring-object appearance with relative offsets. This provides a concrete mechanism for distinguishing similar objects using a reference to another object rather than the target's category alone.

8. Simple Online and Realtime Tracking

   Tracking adds temporal state and identity association to framewise detection. SORT predicts bounding-box motion with a Kalman filter, matches current detections to predicted tracks with overlap-based costs and the Hungarian assignment method, then updates or removes tracks. Detector quality strongly affects the resulting trajectories. A track identifier is a hypothesis maintained through time, not direct evidence that every associated box depicts the same object.

9. Dense-Captioning Events in Videos

   Krishna, Hata, Ren, Fei-Fei and Niebles's 2017 dense-captioning work asks for multiple event intervals and a description of each, including overlapping events. This changes the output from one broad video caption to localized accounts of what happened when. Their system proposes events at different temporal scales and uses surrounding event context when generating descriptions. The accompanying ActivityNet Captions dataset supports evaluating both localization and language.

10. Fast, efficient, reliable: Artificial intelligence in BMW Group Production

    BMW's July 15, 2019 account describes visual inspection in production. At Dingolfing, a live image of a vehicle badge is checked against order data, with mismatches sent to the final-inspection team. Another application distinguishes fine sheet-metal cracks from harmless dust or oil that earlier camera checks sometimes flagged. Employees collect component images from different angles and mark deviations to build task-specific training examples.

11. OpenCV: TextDetectionModel and TextRecognitionModel

    The documented text-spotting pipeline first detects text regions in the full image. Region vertices then drive a geometric transform and crop, and the resulting region image is passed to a text recognizer. Detection answers where text is; recognition predicts its character sequence. Consequently, missed regions never reach recognition, and inaccurate crops or rectification can damage the recognizer's input even when the recognition model itself is unchanged.

12. Foundations of Computer Vision: Imaging

    Pixels depend jointly on illumination, surface reflectance, orientation, and viewing geometry. Under the simplified Lambertian model, reflected intensity is proportional to albedo times incident illumination times the surface-normal/light-direction dot product. Perspective projection maps camera coordinates to x=fX/Z and y=fY/Z. Consequently, different combinations of reflectance and illumination, or object size and distance, can produce the same image. An opaque foreground surface also prevents observation of surfaces behind it. These are missing-evidence ambiguities, not merely classification errors.

13. Physically Based Rendering: Film and Imaging

    A sensor integrates incident light over pixel area and exposure time; color responses additionally integrate wavelength against channel sensitivity. Common sensor mosaics measure one filtered color per photosite, and demosaicing estimates colocated RGB channels. Longer exposure gathers more photons but mixes moving scene positions into blur; larger apertures gather light while reducing depth of field. Photon shot noise follows a Poisson model: for mean count μ, variance is μ and relative standard deviation is 1/sqrt(μ). Digital gain amplifies existing measurements and noise rather than collecting more photons.

14. OpenCV: Image File Reading and Writing

    OpenCV decodes file or memory-buffer contents into an image matrix; unsuccessful decoding can return an empty matrix. Its default color-reading convention stores channels in BGR order. The documented imread behavior applies embedded EXIF orientation unless IMREAD_IGNORE_ORIENTATION or IMREAD_UNCHANGED is supplied. Decoder choices can also affect pixel values: the documentation identifies platform color management and codec-dependent grayscale conversion as examples.

15. Foundations of Computer Vision: Image Sampling and Aliasing

    Sampling stores a continuous image at discrete spatial positions. Frequencies above the representable band can alias into misleading lower-frequency patterns, so distinct continuous images can yield identical samples. For an ideally band-limited signal, exact reconstruction requires sampling frequency greater than twice its maximum frequency and ideal reconstruction filtering. Low-pass filtering before downsampling reduces aliasing by removing unsupported detail. Increasing the dimensions of an already sampled image interpolates values; it does not uniquely recover the discarded detail.

16. Slicing Aided Hyper Inference and Fine-tuning for Small Object Detection

    SAHI processes overlapping image crops separately, then maps and merges their detections into the original image. This preserves a larger object footprint at the detector input than resizing the entire large image. Smaller crops can split large objects, motivating optional full-image inference. A constructed resize example is a 24-pixel-wide object in a 3840-pixel-wide image becoming four pixels wide after resizing to width 640; a 640-pixel crop processed without resizing retains its 24-pixel width.

17. Machine Perception of Three-Dimensional Solids

    Lawrence Gilman Roberts’s 1963 MIT thesis describes a program that processes a photograph into a line drawing, transforms the drawing into a three-dimensional representation, and displays the resulting structure from another viewpoint with hidden lines removed. Its geometric interpretation assumes perspective projection, objects obtainable by transforming known three-dimensional models, and support from other visible objects or a ground plane. The thesis addresses collections of planar-surfaced objects, not unrestricted natural scenes.

18. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information

    Marr proposed the 2½-D sketch as a viewer-centered representation of visible surfaces, combining information from stereo, shading, motion and other visual processes before identifying objects. It describes surface geometry relative to the observer rather than a complete object from every viewpoint. This addressed a difficulty with directly segmenting meaningful objects: strong intensity changes can arise from illumination, while meaningful surface boundaries may have weak intensity changes. Marr's retrospective dates the idea to autumn 1976 and its first published appearance to 1978.

19. Determining Optical Flow

    Horn and Schunck's 1981 work addressed how to estimate image motion when local brightness change supplies insufficient constraints. They added a preference for smoothly varying flow and solved the resulting equations iteratively. Synthetic sequences supplied known motion for comparison, and occluding boundaries produced particularly large errors. Their rotating uniformly shaded sphere example also shows that physical motion can occur without corresponding movement of the visible brightness pattern.

20. Handwritten Digit Recognition with a Back-Propagation Network

    LeCun and colleagues at AT&T Bell Laboratories trained a digit recognizer directly from normalized images instead of relying on a large hand-designed feature-extraction stage. Local connections, shared filter weights and averaging/subsampling constrained what the network could learn and reduced its parameter count. The task used isolated digits from real postal material, with acquisition and preliminary segmentation performed upstream.

21. Gradient-Based Learning Applied to Document Recognition

    LeCun, Bottou, Bengio and Haffner's November 1998 paper presents LeNet-5 as a trainable hierarchy for handwritten-character recognition. Local convolutions share weights across locations, subsampling reduces spatial resolution, and later layers combine the resulting features for classification. Its original subsampling units include learned scale and bias parameters, unlike an ordinary parameter-free average-pooling operation. The paper compares recognition methods on a common handwritten-digit task and develops systems that combine trainable components.

22. Distinctive Image Features from Scale-Invariant Keypoints

    A local descriptor encodes the appearance around an image location as a numeric vector for matching. Direct patch intensities are sensitive to misalignment; SIFT instead summarizes gradients around a scale- and orientation-normalized keypoint. For the Gaussian-smoothed image L, define dx=L(x+1,y)-L(x-1,y), dy=L(x,y+1)-L(x,y-1), magnitude sqrt(dx²+dy²), and orientation atan2(dy,dx). Rotate sampling coordinates and orientations relative to the keypoint orientation. Gaussian-weighted magnitudes vote into orientation histograms, with interpolation across spatial and orientation bins. Concatenating 4×4 histograms with eight bins gives 128 components. Normalize to unit length, cap components at 0.2, then renormalize. Differences remove uniform additive brightness offsets; normalization cancels uniform positive contrast scaling for nonzero gradients. These are prescribed computations, not filters optimized through task-loss gradients.

23. The PASCAL Visual Object Classes (VOC) Challenge

    Everingham and colleagues describe the VOC challenge, begun in 2005, as shared images, annotations and evaluation procedures for recognition in varied natural scenes. Its classification task allows multiple object categories in one image; detection additionally requires their locations. The paper documents VOC 2007's eleven-point interpolated average precision and treatment of duplicate detections and difficult objects. It also explains a segmentation-metric change after the earlier scoring procedure could reward unhelpful predictions, illustrating how benchmark rules evolve with discovered weaknesses.

24. ImageNet: A Large-Scale Hierarchical Image Database

    Jia Deng and colleagues' CVPR 2009 paper addressed the shortage of large, diverse, reliably labeled image collections. ImageNet organized images using WordNet synsets: groups of synonymous words representing a particular concept. Web searches supplied candidates and human verification checked their membership. The reported snapshot contained approximately 3.2 million images across 5,247 synsets. Its experiments examined recognition and use of the hierarchy, making dataset construction part of the technical contribution rather than merely supplying more files.

25. ImageNet Classification with Deep Convolutional Neural Networks

    The 2012 AlexNet work combined large labeled image collections, learned convolutional features, GPU computation and regularization to address recognition across many variable object categories. Its ILSVRC-2012 submission achieved 15.3% top-five test error, compared with 26.2% for the second-place entry. That submission averaged seven networks, including two pretrained on a larger ImageNet collection.

26. Two-Stream Convolutional Networks for Action Recognition in Videos

    Simonyan and Zisserman's 2014 two-stream model separates appearance from motion. One convolutional network processes RGB images and benefits from image pretraining; another processes stacked horizontal and vertical optical-flow fields from adjacent frames. Combining their scores lets motion complement scene and object appearance without asking a small video dataset to teach the entire motion representation from raw pixels. The paper evaluates this combination on UCF101 and HMDB51 action-recognition tasks.

27. Fully Convolutional Networks for Semantic Segmentation

    The FCN work adapted pretrained classification networks into spatial prediction networks by interpreting fully connected layers as convolutions, producing coarse category maps rather than one image-level result. In-network upsampling and connections combining coarse semantic features with finer appearance features supported pixel-level segmentation. Whole-image computation shared work across overlapping regions instead of independently processing every patch.

28. Deep Residual Learning for Image Recognition

    He, Zhang, Ren and Sun at Microsoft Research addressed a problem in which deeper ordinary networks could have higher training error. Their residual block adds a learned transformation to a shortcut input, y=F(x)+x, allowing a block to learn a change relative to identity. In the paper's ImageNet comparison, the 34-layer residual network achieved 25.03% top-one validation error versus 28.54% for its plain counterpart under ten-crop evaluation, alongside improved training optimization.

29. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

    For an H×W image with C channels and square P×P nonoverlapping patches, N=HW/P² when both dimensions are divisible by P. Each flattened P²C patch is linearly projected to a D-dimensional token; learned positions and a classification token are added. In each attention head, Q, K, and V are learned projections, A=softmax(QKᵀ/sqrt(d)), and AV mixes value vectors using input-dependent weights. Position embeddings provide spatial identity absent from content-only attention. Full attention forms an approximately N×N matrix: doubling both image dimensions at fixed P gives four times as many patches and sixteen times as many pairwise entries. Higher-resolution transfer interpolates patch-position embeddings.

30. Learning Transferable Visual Models From Natural Language Supervision

    CLIP trains separate image and text encoders so matching image-caption pairs have higher similarity than mismatched pairs. At inference, descriptions of candidate labels can be embedded and compared with an image, enabling classification without training a new label-specific head. The learned similarity space also supports retrieval. This is alignment from paired data, not a symbolic database of visual facts; candidate descriptions and data coverage affect what comparisons mean. Zero-shot transfer evaluates a different adaptation regime from training on the target dataset.

31. Segment Anything

    SAM separates an image encoder, a prompt encoder, and a mask decoder. Points, boxes, or masks can specify the intended region, while the decoder predicts pixel-level masks using a reusable image embedding. A prompt can be ambiguous: one point might refer to a whole object or one of its parts. The system therefore predicts multiple candidate masks and ranks them with an estimated overlap score. The authors report missed fine structures, disconnected components, and imperfect boundaries, so promptable segmentation remains a prediction to validate.

32. An Iterative Image Registration Technique with an Application to Stereo Vision

    Bruce Lucas and Takeo Kanade's 1981 method estimates how to align image regions using their spatial intensity gradients. Iterative local updates replace an exhaustive search over possible alignments. Sharing an alignment across a region supplies constraints unavailable from an isolated pixel. The paper extends the formulation beyond translation and discusses smoothing and coarse-to-fine processing to accommodate larger initial displacement. These methods trade fine detail for a wider range of recoverable motion.

33. CS231n: Convolutional Networks

    A convolutional layer slides filters across local neighborhoods, summing weighted inputs across spatial offsets and input channels to produce output channels. Reusing filters shares parameters across positions. Unlike SIFT's prescribed gradient histograms, these filter weights are optimized by gradient descent against a training objective. At one position, the output-channel vector can be interpreted as a learned local descriptor; deeper layers describe larger neighborhoods. For input width W, kernel k, padding p, stride s, and unit dilation, output width is floor((W+2p-k)/s)+1. Pooling summarizes neighborhoods, commonly by their maximum, without learned weights. For sequential undilated layers, receptive-field bookkeeping is j_l=s_l j_(l-1), r_l=r_(l-1)+(k_l-1)j_(l-1), starting with j_0=r_0=1; j measures input-pixel spacing between outputs and r their theoretical support.

34. PyTorch: Training a Classifier

    The CIFAR-10 example converts RGB images into normalized tensors and batches them as B×3×32×32, paired with B integer class labels. Learned convolution and linear weights transform inputs through convolution, ReLU, pooling, and flattening into B×10 logits. Cross-entropy compares logits with target classes; for one example it is -log softmax(logits)[target]. Training clears old gradients, runs the forward pass, calls loss.backward() to accumulate parameter derivatives by the chain rule, then optimizer.step() updates parameters using SGD with momentum. Fixed-parameter inference retains preprocessing and the forward pass, while target labels, training loss, backpropagation, and optimizer updates are unnecessary. The example uses no_grad() during testing.

35. Foundations of Computer Vision: Convolutional Neural Nets

    Translation equivariance means translating an input translates its feature map: f(Tx)=T f(x). Invariance means the output stays unchanged: f(Tx)=f(x). Shared convolution supports equivariance; pooling can suppress sensitivity to feature position within its pooling region. Global pooling removes spatial position more extensively. Downsampling reduces spatial resolution while deeper layers combine larger neighborhoods and often encode more category-related features. This exchanges detailed localization for contextual coverage rather than creating new image evidence.

36. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

    Liu and colleagues' 2021 Swin Transformer addressed the need for multiscale visual features and manageable attention computation on large images. Patch merging progressively reduces spatial resolution while increasing feature width. Attention operates within local windows, with alternating shifted partitions allowing information to cross previous window boundaries. This produces a hierarchy usable by detection and segmentation heads. The paper evaluates classification, detection and segmentation and studies the contribution of shifted windows.

37. How Transformers Finally Ate Vision

    The speaker argues that masked-image reconstruction can recover useful spatial behavior while allowing specialized architectural biases to be removed.

38. How Transformers Finally Ate Vision

    Frozen pretrained features can be evaluated with a learned linear projection, isolating how much useful information the representation already contains.

39. PyTorch: BCEWithLogitsLoss

    The documented multilabel objective applies a sigmoid and binary cross-entropy separately for each sample-class entry, with targets shaped like the logits. Several classes can therefore have positive targets for one image; the outputs are not constrained to allocate one unit of probability across mutually exclusive categories. The implementation combines sigmoid and loss computation for numerical stability.

40. 120k players in a week: Lessons from the first viral CLIP app: Joseph Nelson

    Paint.wtf uses CLIP (Contrastive Language-Image Pre-Training) to rank drawings by cosine similarity between prompt and image embeddings.

41. 120k players in a week: Lessons from the first viral CLIP app: Joseph Nelson

    CLIP's ability to recognize written text allowed users to score highly by writing the requested scene instead of drawing it.

42. Vision AI in 2025 — Peter Robicheaux, Roboflow

    Conceptual recognition does not guarantee precise spatial perception, and a wrong visual answer can be accompanied by fabricated supporting details.

43. Towards Open Set Deep Networks

    A closed-set classifier distributes its output over known categories and always has a highest-scoring category, including for inputs outside that vocabulary. Open-set recognition adds the possibility of rejecting unknown categories. The authors evaluate thresholded softmax and their OpenMax method on known, unrelated and fooling images, showing that rejecting low-confidence predictions alone does not reliably reject unknown inputs in those experiments.

44. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

    Faster R-CNN shares convolutional image features between a region-proposal network and a detector that consumes its proposals. At each feature-map location, the proposal network predicts object-versus-background scores and box adjustments relative to reference rectangles called anchors. Anchors provide different scales and aspect ratios. The resulting rectangular candidates are intermediate proposals, rather than final class-specific detections.

45. You Only Look Once: Unified, Real-Time Object Detection

    Original YOLO predicts boxes and category information in one network evaluation using an image grid. The cell containing an object's center is responsible for that object; each cell predicts a fixed number of boxes and associated confidence values. Its training confidence target combines object presence with box overlap, and class-specific scores combine that confidence with conditional class predictions. The paper identifies difficulty precisely localizing small objects.

46. Feature Pyramid Networks for Object Detection

    ResNet backbone outputs C2–C5 have input-relative strides 4, 8, 16, and 32. FPN projects these through lateral 1×1 convolutions, upsamples the coarser top-down map by two, and adds maps at matching spatial sizes. A final 3×3 convolution yields each P-level. P2–P5 retain the corresponding C-level resolutions, with 256 channels each in the paper. They are newly combined representations, not unchanged copies of C2–C5. Coarse semantic information reaches fine grids while lateral connections contribute better-localized features. Detection heads operate across the pyramid; the RPN variant adds P6 by stride-two subsampling of P5.

47. Torchvision Non-Maximum Suppression

    For valid continuous XYXY boxes A and B, intersection width is max(0,min(Ax2,Bx2)-max(Ax1,Bx1)), with analogous height. Intersection area I is their product and IoU=I/(area(A)+area(B)-I). NMS keeps high-score candidates and iteratively removes lower-score boxes whose IoU with a retained higher-score box exceeds the threshold. Suppression compares predictions with predictions, not ground truth. Torchvision returns retained indices in descending score order.

48. End-to-End Object Detection with Transformers

    DETR predicts a fixed set of query outputs, each containing class probabilities and a box. During training, Hungarian matching finds a minimum-total-cost one-to-one assignment between predictions and ground-truth objects, using class compatibility and box similarity. Matched pairs receive classification and box losses; unmatched predictions learn the no-object category. The box loss combines L1 distance with generalized IoU. Unlike inference-time NMS, this assignment compares predictions with labels to define supervision rather than deleting overlapping candidates. The original set-prediction design avoids an NMS postprocessing stage.

49. COCO Official Detection Evaluator

    For each class and IoU threshold, COCO sorts detections by score and greedily matches each to an eligible ground-truth object in the same image. Ordinary ground truths match once; additional duplicates become false positives. Accumulated TP and FP give precision=TP/(TP+FP) and recall=TP/number_of_nonignored_ground_truths. Sweeping the score-ranked list produces operating points; AP averages interpolated precision at 101 recall levels. Standard COCO AP additionally averages over classes and ten IoU thresholds, 0.50 through 0.95 by 0.05, using the 100-detection cap for its main summary. Filtering out low scores before evaluation can truncate achievable recall.

50. Panoptic Segmentation

    Panoptic segmentation assigns each pixel a semantic category and, for countable object categories, an instance identifier. It combines individual 'things', such as cars, with 'stuff', such as sky, whose instance identifiers are irrelevant. The format permits only one category-instance assignment per pixel, so segments do not overlap. Ambiguous or out-of-vocabulary pixels may receive a void label. The paper explicitly concerns visible regions rather than completing hidden object extents.

51. U-Net: Convolutional Networks for Biomedical Image Segmentation

    Original U-Net repeatedly applies two unpadded 3×3 convolutions and stride-two 2×2 max pooling, doubling channels during descent. The expanding path upsamples, applies a 2×2 up-convolution, concatenates cropped features from the matching contracting level, and applies further convolutions. Cropping aligns spatial supports because unpadded convolutions shrink maps. A final 1×1 convolution produces per-pixel class logits. Pixelwise softmax and weighted cross-entropy supervise the map: L=-Σ_x w(x) log p(target(x)|x). Weights compensate class frequencies and emphasize boundaries between touching cells. The output covers an interior region; overlap tiling and mirrored border context support larger images.

52. Mask R-CNN

    He, Gkioxari, Dollár and Girshick's 2017 Mask R-CNN extends Faster R-CNN with a parallel mask-prediction branch for each proposed object. Its RoIAlign operation avoids rounding region coordinates and uses bilinear interpolation to sample features, preserving alignment needed for pixel-level predictions. The mask branch predicts binary masks separately from object classification; the selected class determines which mask is used. Experiments demonstrate the importance of alignment when extending box detection to instance segmentation.

53. Magic Editor Under the Hood: Weaving Generative AI into a Billion-User App

    A U-Net portrait segmentation model can produce a usable mask while missing fine hair strands; post-model image understanding can refine those boundaries.

54. SAM 3: Segment Anything with Concepts

    SAM 3 adds promptable concept segmentation: a short noun phrase, image exemplars, or both specify a concept whose matching instances should be detected, segmented and tracked. A positive box around one dog can request all dogs, whereas an instance-selection prompt requests that particular object. Positive and negative exemplars refine the concept; clicks can refine individual masks. The system combines a detector that discovers objects with a tracker that propagates identities. Its task definition explicitly recognizes ambiguous words, subjective attributes and uncertain object boundaries.

55. Detectron2: Use Custom Datasets

    Detectron2 represents keypoints with continuous image coordinates ranging from zero to image width or height. Its documentation distinguishes these from COCO's integer pixel indices and states that loading COCO keypoints adds 0.5 to their coordinates. Dataset metadata separately specifies keypoint names and the name pairs exchanged during horizontal augmentation.

56. Foundations of Computer Vision: Camera Modeling and Calibration

    A calibrated pinhole model relates a homogeneous world point X to image coordinates through λx=K[R|t]X. Intrinsics K describe focal scaling and principal point; extrinsics R,t transform world coordinates into camera coordinates. Dividing by the final homogeneous coordinate gives pixel coordinates. Calibration estimates these parameters from known geometric observations. A pixel then constrains a viewing ray, not a unique depth; correspondence to additional calibrated views can constrain the same scene point further.

57. A Flexible New Technique for Camera Calibration

    Zhang's calibration method addressed the expense and complexity of specialized three-dimensional calibration apparatus. It uses a known planar pattern viewed at different orientations, without requiring known motion between views. A printed pattern attached to a reasonably flat surface supplies geometric information; the method estimates camera parameters and radial distortion, followed by numerical refinement. This offers a practical route between elaborate calibration objects and calibration from scene images alone.

58. OpenCV: Epipolar Geometry

    A point in one camera image constrains its corresponding point in another image to an epipolar line. In homogeneous pixel coordinates this is l'=Fx, with a valid match satisfying x'ᵀFx=0. The fundamental matrix incorporates camera intrinsics and relative geometry; the essential matrix expresses the relation in calibrated coordinates. This reduces correspondence search from a two-dimensional image to a line, but still requires finding the correct matching observation.

59. Depth Anything V2

    Depth Anything V2's base models predict inverse depth with scale-and-shift ambiguity; metric depth requires a separately adapted model. The authors train a teacher on synthetic depth labels, use it to label real images, and train students on those generated targets. Their DA-2K evaluation asks which of two image points is closer, while separate experiments evaluate metric depth. The paper explicitly distinguishes sparse relative-order judgments from the dense precision needed for reconstruction.

60. Foundations of Computer Vision: Multiview Geometry and Structure from Motion

    Structure from motion jointly estimates scene structure and camera viewpoints from corresponding image observations. Unlike calibrated stereo, camera motion can be an unknown to solve alongside geometry. Repeated observations connect the unknown camera and point variables; optimization seeks a reconstruction whose projections agree with measurements. This requires assumptions about shared scene structure and sufficient informative views, rather than simply interpreting each frame independently.

61. Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection

    Shilong Liu and colleagues' 2023 Grounding DINO connects text and image features at several stages of a detector. A feature enhancer exchanges information between modalities, language-guided selection chooses image queries, and a decoder attends to both representations before predicting boxes and text associations. Training aligns object queries with text tokens alongside box losses. The paper evaluates detection from category descriptions and localization from referring expressions, extending the input contract beyond a fixed numerical class list.

62. CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning

    CLEVR pairs rendered scenes with recorded object attributes, positions and executable question programs. These programs combine operations such as selecting objects by attributes, following spatial relationships, counting and comparing. This separates the required reasoning operations from a question's wording. Its spatial relations use an explicit camera-dependent convention: projecting the viewing direction onto the ground plane defines behind, with other directions defined correspondingly. Scene graphs can replace image perception with known object information when diagnosing the reasoning component.

63. GRES: Generalized Referring Expression Segmentation

    GRES extends referring-expression segmentation to descriptions matching one, multiple or no objects. Its gRefCOCO samples include a mask covering all intended targets and an explicit no-target label; absent targets require an empty mask. Evaluation measures target absence separately from mask overlap. A published failure involves correctly recognizing a seated person and nearby bed but selecting the person for a description requiring sitting on the bed, although the person is on a chair. Recognizing the components did not establish their stated relationship.

64. Visual Instruction Tuning

    Original LLaVA extracts CLIP ViT-L/14 grid features Z and applies a learned linear projection H=WZ into the language model's embedding width. Visual embeddings join instruction text in the autoregressive context; generated answer tokens follow p(a|image,instruction)=Π_t p(a_t|image,instruction,a_<t). Training first freezes both vision encoder and language model while learning the projection from image-caption pairs. The second stage keeps the vision encoder frozen but updates the projection and language model on visual instruction-response data. The bridge therefore learns both embedding compatibility and how visual evidence should condition language responses.

65. Moondream: how does a tiny vision model slap so hard?

    Define image understanding separately from general reasoning, then select data and benchmarks for that narrower contract.

66. Moondream: how does a tiny vision model slap so hard?

    The speaker warns that blindly training on a more capable model's answers can teach plausible elaboration rather than grounded image understanding.

67. A Simple Framework for Contrastive Learning of Visual Representations

    SimCLR creates a positive pair by independently augmenting the same image, using cropping and resizing, color distortion and blur. An encoder maps the views to representations; a projection head maps those representations into the space where the contrastive objective operates. Other images' augmented views supply competing examples. After pretraining, the projection head is discarded and encoder representations are used for downstream tasks. The authors experimentally study how augmentation composition changes representation quality.

68. Foundations of Computer Vision: Training for Robustness and Generality

    Label-preserving augmentation assumes y(T(x))=y(x). This is task-dependent: mirroring can preserve a scene category while changing a character's identity. When targets have spatial structure, transform image and target together so y(Tx)=T_y(y(x)); a segmentation crop must crop the label map at the same coordinates. Augmentation teaches selected invariances or equivariances by adding transformed training examples. It cannot repair an invalid label-preservation assumption or supply an independent evaluation sample.

69. Masked Autoencoders Are Scalable Vision Learners

    MAE randomly removes image patches, commonly 75% in the paper, and sends only visible patch embeddings with positions through the encoder. A smaller decoder receives encoded visible patches plus mask tokens and positional information for all locations. It predicts patch pixels and minimizes mean squared reconstruction error only on masked patches; a variant normalizes target pixels within each patch. The decoder is discarded for downstream recognition. Unlike supervised classification, targets come from hidden portions of the input rather than category annotations. Unlike paired image-text contrastive learning, the objective reconstructs image content rather than distinguishing matched from mismatched image-caption pairs.

70. Emerging Properties in Self-Supervised Vision Transformers

    Caron and colleagues' 2021 DINO method trains a student to match a teacher's output distributions across different views of an image without human category labels. The teacher follows an exponential moving average of student parameters; centering and sharpening its outputs counter different collapse tendencies. Local student crops must agree with teacher views covering more of the image. Experiments evaluate reusable recognition features and show object-related structure in transformer attention maps.

71. DINOv2: Learning Robust Visual Features without Supervision

    DINOv2 combines image-level and patch-level supervision derived from images. A student predicts a teacher's outputs across different crops; for masked student patches, it predicts the teacher's corresponding unmasked patch outputs. The teacher follows an exponential moving average of student parameters. Unlike pixel reconstruction, these targets are learned feature distributions. Downstream experiments freeze the encoder and train readouts for classification, segmentation and depth. Its simplest segmentation readout predicts classes from patch features and upsamples the coarse map; a separate experiment trains an adapter and segmentation head while retaining the frozen backbone.

72. DINOv3

    During extended DINOv3 pretraining experiments, image-classification accuracy continued improving while segmentation performance from patch features declined. Patch similarity maps also became less localized. Gram anchoring addressed this by encouraging the student's pairwise patch-feature similarities to resemble those of an earlier teacher with stronger dense features. This preserves relationships among local representations without requiring their individual vectors to remain unchanged. The authors report improved dense-task performance after introducing this objective.

73. Fine-Tuning Can Distort Pretrained Features and Underperform Out-of-Distribution

    Linear probing fits a linear task head while freezing pretrained features; full fine-tuning updates both. The authors compare these routes across ten distribution-shift datasets and find that full fine-tuning can improve in-distribution accuracy while reducing out-of-distribution accuracy relative to probing. Their feature-distortion explanation is analyzed in a simplified two-layer linear setting. They also test initializing full fine-tuning from a fitted linear probe.

74. Roboflow100-VL: A Multi-Domain Object Detection Benchmark for Vision-Language Models

    RF100-VL deliberately selects detection tasks where class names alone can be insufficient, including ambiguous names and material categories. It supplies reviewed annotation instructions and visual examples explaining the intended concepts. Its 10-shot subsets contain ten object instances per class, rather than necessarily ten images. Evaluation distinguishes prompting frozen models from parameter fine-tuning. It uses the COCO evaluator with a 500-detection limit because some images contain more than 100 objects, and documents how referential-grounding and object-detection protocols produce different reported results.

75. Vision AI in 2025 — Peter Robicheaux, Roboflow

    A small detector trained on a few examples can outperform a zero-shot specialist; adapting the specialist can reverse that ranking.

76. Semantic Amodal Segmentation

    The paper distinguishes segmentation of visible pixels from amodal segmentation, which annotates an object's full estimated extent, including hidden portions. Its annotation scheme adds semantic names and partial depth ordering; annotating occluding regions lets visible and occluded portions be distinguished. Thus the same image can support different supervised targets depending on whether the task requests observed extent or an inferred completion.

77. LVIS: A Dataset for Large Vocabulary Instance Segmentation

    LVIS separates verified category presence, verified absence and categories not assessed in an image. Its federated evaluation judges a category only where suitable positive or negative annotations exist. A further flag records whether all instances of a present category were annotated. On images flagged non-exhaustive for that category, the evaluator measures recall on labeled instances without counting unmatched predictions as false positives. The annotation process separately checks mask quality, completeness and negative labels. Missing annotation therefore has a different meaning from verified background or absence.

78. Google ML Crash Course: Class-Imbalanced Datasets

    Class imbalance means some labels occur much more frequently than others. A classifier can obtain high overall accuracy while barely learning rare classes; minibatches may contain too few minority examples. Resampling changes the class frequencies encountered during training, while loss weights change each example's contribution to optimization. The documented majority-downsampling approach compensates with inverse sampling weights. Rebalancing choices affect the effective training distribution and require validation; aggregate accuracy alone is insufficient to describe rare-class performance.

79. Torchvision Segmentation Reference Transforms

    The segmentation reference resizes the image with antialiasing but resizes its target using nearest-neighbor interpolation. It shares crop parameters and flip decisions between image and target, converts target arrays to integer tensors, and normalizes only the image. This illustrates why categorical targets need different numerical treatment from image intensities while preserving the same spatial transformation.

80. COCO-WholeBody: Annotation File Format

    The COCO-WholeBody specification preserves COCO's body-keypoint fields: visibility zero means unannotated, one means annotated but not visible, and two means annotated and visible. Unannotated coordinates are zero. The number of annotated body keypoints counts both positive visibility values. Consequently, annotation availability and physical visibility are distinct target properties.

81. Detectron2: Keypoint Annotation Transformations

    Detectron2 applies the image's coordinate transformations to keypoint positions, then checks them against the transformed image bounds. Points outside those bounds become unlabeled with visibility zero; all unlabeled points have their coordinates zeroed. An odd number of horizontal flips additionally permutes entire keypoint records using a supplied left/right mapping. The mapping comes from keypoint names and paired opposite-handed names; unpaired names retain their positions in the ordering. Surviving points otherwise retain their visibility values. Thus updating coordinates alone misses changes to annotation eligibility and semantic identity.

82. FFmpeg: AVFrame Struct Reference

    FFmpeg represents a frame's presentation timestamp separately from its decoded image data. The pts field specifies when the frame should be presented, in time-base units; pkt_dts records decoding-related timing. Frame timing therefore needs an explicit interpretation rather than treating an array index as elapsed time.

83. Retro-Actions: Learning ‘Close’ by Time-Reversing ‘Open’ Videos

    Video transformations can preserve a class, exchange it with another class, or create an invalid or previously unrepresented action. Horizontal reflection changes directional gestures such as swiping right; time reversal can turn covering into uncovering. Other reversals create implausible physics, such as an object spontaneously rising from the floor. The authors establish transformation rules through class semantics and visual inspection, then evaluate transformed training examples on Jester and Something-Something. Their opening-versus-closing example shows why temporal order carries information that an unordered collection of the same frames lacks.

84. Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset

    Carreira and Zisserman's 2017 I3D approach turns an image network into a video network by adding a temporal dimension to convolution and pooling kernels. Repeating a pretrained spatial filter across time and scaling its weights initializes the new filter to preserve its response on a repeated still image. Video training then learns space-time features. This provides a route from image pretraining to deeper video architectures, alongside the paper's larger Kinetics action dataset and transfer experiments.

85. Is Space-Time Attention All You Need for Video Understanding?

    Bertasius, Wang and Torresani's 2021 TimeSformer represents video as patches associated with spatial positions and frames. Its divided-attention block first exchanges information across time at matching patch positions, then across spatial positions within each frame. This avoids applying joint attention to every space-time pair in one operation. Experiments compare spatial-only, joint and divided attention, finding substantially different behavior on tasks requiring temporal information.

86. HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking

    HOTA separates finding objects from associating them consistently over time. Detection errors include missing objects and extra detections; association errors include splitting one reference trajectory across predicted identities or merging different reference trajectories under one identity. It matches detections within frames, assesses trajectory agreement, and averages across localization thresholds. Its component metrics allow systems with similar overall scores to be examined for different failure types.

87. Simple Online and Realtime Tracking with a Deep Association Metric

    Wojke, Bewley and Paulus's 2017 Deep SORT combines predicted box motion with learned appearance descriptors to associate detections over time. A Kalman filter predicts position and uncertainty; appearance comparisons help distinguish candidates when motion alone is unreliable. A matching cascade prioritizes recently observed tracks, while track initialization and deletion manage arrivals and prolonged misses. Its pedestrian-tracking experiments demonstrate why finding an object in each frame and maintaining its identity are separate problems.

88. SAM 2: Segment Anything in Images and Videos

    SAM 2 extends promptable segmentation across video by conditioning each frame's features on stored predictions and prompted frames. Points, boxes or masks can select a target, and later prompts can correct its propagated masks. The model also predicts whether the target is present, allowing frames without a visible target. Its evaluation distinguishes repeated offline passes from a single forward pass with interactions.

89. Online Action Detection

    Online action detection predicts whether an action is occurring using only information available up to the current frame. Unlike classifying a pretrimmed action clip, the system must handle unknown action boundaries and long stretches of background or unrelated activity. The TVSeries benchmark annotates conditions including occlusion, camera movement, shot changes, unusual viewpoints and missing action beginnings or endings.

90. Ego4D: Around the World in 3,000 Hours of Egocentric Video

    Grauman and the Ego4D consortium's CVPR 2022 paper includes tasks for understanding object state changes in video recorded from a camera wearer's perspective. It distinguishes whether a change occurred, when it began, and which object changed. Annotations identify precondition, change and postcondition frames, together with relevant objects, hands and tools. This separates recognizing an action from identifying its visible consequence: different tools or motions can produce a similar material change.

91. SlowFast Networks for Video Recognition

    SlowFast separates slowly changing appearance from more rapidly changing motion using two connected video-processing pathways. In its published example, both pathways cover a 64-frame clip: the slow pathway receives four frames and the fast pathway receives 32. The fast pathway uses fewer feature channels but preserves its temporal resolution until final pooling. Temporal convolutions process ordered frame neighborhoods before the resulting features are summarized for classification.

92. Online Detection of Action Start in Untrimmed, Streaming Videos

    Zheng Shou and colleagues introduced an action-start evaluation protocol in 2018. Predictions contain a time, class and confidence. A prediction is correct only when its class matches and its time is within a chosen absolute offset of the annotated action start; duplicate detections of the same start are not allowed. The protocol computes average precision by class and then averages classes. Experiments simulate sequential frame arrival using untrimmed video. This distinguishes action-start detection from per-frame classification and full-interval localization.

93. VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding

    Hu Xu and colleagues' EMNLP 2021 VideoCLIP learns compatible video and text representations from instructional videos and their transcripts. Temporally overlapping clips provide positive pairs, while retrieval supplies difficult negative examples from similar videos. A video transformer processes visual features; pooled outputs support clip retrieval, while token-level representations support finer action and step tasks. The paper evaluates retrieval, answer selection, action segmentation and step localization without task-specific supervised training.

94. TempCompass: Do Video LLMs Really Understand Videos?

    TempCompass's 2024 evaluation constructs related videos whose static content is similar but whose temporal meaning differs. Its tests cover actions, speed, direction, attribute changes and event order. Reordering event clips, for example, changes which event happened first while retaining the constituent events. Multiple-choice questions, yes/no questions, caption matching and caption generation expose different response behaviors. This supplies a diagnostic design for testing whether answers follow temporal information rather than recognizable objects alone.

95. LLaVA-OneVision: Easy Visual Task Transfer

    Bo Li and colleagues' 2024 LLaVA-OneVision processes individual video frames with a vision encoder and projects their features into a language model's input space. Its video representation resizes frames to the encoder's base resolution and reduces feature-token counts through bilinear interpolation, allowing more frames within a limited budget. Training combines image, multiple-image and video tasks, with experiments demonstrating transfer between these scenarios. The design makes spatial detail per frame and temporal coverage separate allocation choices.

96. Selective Classification for Deep Neural Networks

    A selective classifier combines predictor f with acceptance function g(x)∈{0,1}. Coverage is E[g(x)]; selective risk is E[loss(f(x),y)g(x)]/E[g(x)], the error among accepted examples. Thresholding a confidence score changes the accepted subset and produces a risk-coverage curve. The paper's risk-control procedure uses labeled samples and statistical upper bounds to select a threshold for a target risk with stated failure probability. Choosing a threshold therefore requires a defined loss, representative validation evidence, enough accepted examples, and accounting for threshold search. Confidence alone does not supply those guarantees.

97. WILDS: A Benchmark of in-the-Wild Distribution Shifts

    The iWildCam task tests species classification across camera traps: stationary cameras triggered by heat or motion. Its unseen-camera evaluation separates locations, while its same-camera evaluation separates capture days. Burst images remain together. Camera deployments differ in backgrounds, illumination, viewpoints and species frequencies. The benchmark reports macro F1 alongside average accuracy to expose performance on less frequent classes, and documents label noise when an entire burst receives a species label despite some frames being empty.

98. ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness

    Geirhos and colleagues created images whose object shape and transferred texture suggested different categories. In these cue-conflict experiments, tested ImageNet-trained CNNs favored the texture category more often than human observers. Training the same ResNet-50 architecture on stylized images shifted its choices toward shape. The experiment demonstrates that training data can change which visual cues guide recognition without changing the architecture.

99. ImageNet-trained CNNs are not biased towards texture: Revisiting feature reliance through controlled suppression

    This study distinguishes choosing a cue when cues conflict from relying on that cue when recognizing ordinary images. It evaluates accuracy after transformations intended to suppress texture, color or spatial shape structure. In its selected ImageNet experiment, tested CNNs were more vulnerable to disruption of local shape than to texture suppression. The authors also show that common smoothing operations remove different amounts of shape information, complicating interpretation of a supposedly texture-only intervention.

100. Noise or Signal: The Role of Image Backgrounds in Object Recognition

     The authors construct ImageNet-9 variants that separate or recombine foregrounds and backgrounds. Models can classify above chance using backgrounds alone. Comparing class-consistent replacement backgrounds with randomized backgrounds shows that changing background class signal can reduce recognition accuracy while the foreground remains present. The class-consistent replacement condition helps control for artifacts introduced by image compositing.

101. Benchmarking Neural Network Robustness to Common Corruptions and Perturbations

     ImageNet-C tests classifiers on algorithmically corrupted ImageNet validation images: 15 corruption types, each at five severities, covering noise, blur, weather and digital effects. ImageNet-P instead generates small-change sequences to assess prediction stability. The protocol reserves these corruptions for evaluation rather than training, separating performance under degraded images from ordinary clean-image accuracy.

102. Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift

     The study evaluates uncertainty methods under image corruptions and other distribution shifts, measuring accuracy, calibration, and probabilistic scores. Across its experiments, uncertainty quality generally deteriorates as shift increases. Temperature scaling fitted on an in-distribution validation set can calibrate nearby test data without remaining calibrated under stronger shift. Models with better original-distribution accuracy or calibration do not necessarily retain that ordering under shift. Ensembles perform comparatively well in the studied settings but do not eliminate degradation.

103. Cityscapes Official Pixel-Level Evaluator

     For class c, IoU_c=TP_c/(TP_c+FP_c+FN_c): intersection is correctly predicted c pixels, and union includes pixels labeled or predicted c. Cityscapes accumulates a confusion matrix across evaluated images before computing per-class scores. Ignored ground-truth labels do not contribute false positives for evaluated classes. Predictions of an ignored class at an evaluated ground-truth pixel still contribute a false negative for that ground-truth class. Ignored classes and zero-union classes return NaN, and getScoreAverage excludes NaNs. Mean IoU gives equal weight to each valid evaluated class rather than weighting classes by pixel count.

104. V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation

     For binary prediction and reference arrays, the paper's Dice expression reduces to twice their intersection divided by the sum of their foreground sizes: 2TP/(2TP+FP+FN). True-negative background locations do not enter this foreground-overlap score. The paper motivates a differentiable Dice-based training objective with small foreground structures that can otherwise be overwhelmed by background.

105. Boundary IoU: Improving Object-Centric Image Segmentation Evaluation

     Boundary IoU compares the interior bands near prediction and reference boundaries rather than entire mask areas. The paper studies metric sensitivity by constructing controlled dilation, erosion, displacement, boundary perturbation, simplification and hole errors. These tests expose differences between area agreement and contour agreement. Boundary-band width controls which nearby pixels participate, while errors far from both boundaries can receive little attention from a boundary-only measure.

106. COCO Official Evaluator: computeOks

     For a reference with annotated keypoints, computeOks averages exp(-d_i²/[2(A+epsilon)(2sigma_i)²]) over landmarks whose reference visibility exceeds zero. Here d_i is the predicted-to-reference coordinate distance, A is reference object area, and sigma_i is the configured landmark-specific tolerance. Both reference visibility values one and two participate; predicted visibility values do not enter this calculation. Larger object area or tolerance reduces the penalty for the same pixel displacement. Unlike box or mask IoU, OKS compares corresponding landmark locations rather than intersecting regions. References with no annotated keypoints are marked ignored during preparation; computeOks has a separate expanded-box distance fallback for them.

107. The Fourth Monocular Depth Estimation Challenge

     The 2025 challenge changed its evaluation to align predictions with reference depth using a fitted scale and shift. Disparity predictions were first inverted, and predictions were resized to the reference resolution. Evaluation then included pixelwise depth errors, reconstructed-point-cloud agreement and depth-boundary measures. Because reference data determine the fitted scale and shift, these scores assess performance after alignment rather than autonomous recovery of absolute scale.

108. Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality

     Winoground pairs two images with two captions containing the same words in different orders. Correct matching requires distinguishing how objects, attributes or relations are combined rather than merely recognizing their presence. Text scoring tests caption choice for both images; image scoring tests image choice for both captions; group scoring requires all these comparisons to succeed. Expert annotation and fine-grained tags support examining particular compositional failures.

109. Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking

     Ristani and colleagues' 2016 identity metrics assess how much tracking is correctly identified rather than merely counting identity switches. A global one-to-one matching between reference and predicted trajectories determines correctly identified detections, identity false positives and identity false negatives. IDF1 combines these as 2IDTP/(2IDTP+IDFP+IDFN). Consequently, a long period assigned to the wrong identity can matter differently from a brief switch, even when switch counts coincide.

110. ActivityNet Official Temporal Detection Evaluator

     ActivityNet's detector evaluation consumes a video identifier, action label, start/end interval and score for each prediction. Within each class, predictions are ranked by score and matched to reference intervals in the same video using temporal intersection over union. A reference interval can be matched only once per overlap threshold; unmatched predictions count as false positives. The default summary averages AP across classes and ten temporal-overlap thresholds from 0.50 to 0.95.

111. ActEV Sequestered Data Leaderboard Evaluation Plan

     The 2019 SDL plan evaluates selected video frames. Its time-based false alarm measure sums positive excess system-instance counts over reference-instance counts across frames, divided by duration without the target activity. This measures excess predicted instance-time, not alert counts per hour. Miss probability is the fraction of reference instances left undetected after confidence-sensitive, one-to-one Hungarian matching. Labels must agree. The printed overlap gate requires at least one second when the system interval lasts at least one second; shorter system intervals require overlap covering at least half the reference duration. Unmatched system instances do not directly increase miss probability but can increase false-alarm time. Evaluation is retrospective; runtime factor divides specified processing-call durations by video duration.

112. ActEV 2021 Sequestered Data Leaderboard Evaluation Plan

     The 2021 SDL plan measures runtime using designated processing calls, excluding work before access to test video and instance shutdown. It allows partitioned processing and computes runtime factors separately for electro-optical and infrared video. Under time-limited scoring, exceeding the real-time processing budget causes scoring to treat execution as stopped within the offending partition, with missed detections incurred for that partition and subsequent partitions. This penalizes unfinished processing under a workload budget; it does not measure elapsed time from an individual event's onset to delivery of its alert.

113. scikit-learn: Cross-Validation for Grouped and Time-Dependent Data

     Ordinary cross-validation assumes examples are sufficiently independent. Repeated observations of the same subject can let a model exploit subject-specific features, overstating performance on unseen subjects. GroupKFold keeps each group entirely on one side of a split; StratifiedGroupKFold additionally attempts to preserve class proportions. Time-dependent data may require forward-ordered splits. For vision, grouping frames from the same video, capture session, or individual before partitioning is an application of these principles.

114. WILDS Release Notes: iWildCam v1.0 to v2.0

     The WILDS maintainers documented that the original iWildCam in-distribution split randomly separated images, allowing frames taken seconds apart by the same camera into training, validation and test sets. Dataset v2.0 instead keeps every camera-day together. The release also changed image membership and categories, so comparing versions changes more than the splitting procedure.

115. Google ML Guides: Data Quality and Interpretation

     Collected data are an imperfect measurement of reality: instruments can fail, people can mislabel examples, and reasonable annotators can disagree on ambiguous signals. Sampling processes such as self-selection and survivorship change which examples are represented. For visual systems, camera conditions, chosen locations, and annotation instructions therefore shape both learned targets and measured evaluation performance. Record collection conditions and label definitions, inspect ambiguous cases, and distinguish annotation errors from legitimate disagreement.

116. Magic Editor Under the Hood: Weaving Generative AI into a Billion-User App

     Treat the benchmark as a maintained model regression suite whose examples must continue to reflect real usage.

117. A framework for flexible and reconfigurable vision inspection systems

     The reported automotive inspection station coordinates component rotation, lighting and multiple cameras, capturing 36 frames per camera during a rotation. Image-processing results determine whether a component proceeds or receives manual reinspection. An initial production-line test of 60 components found 15 defective components correctly flagged, four acceptable components falsely flagged, 41 correctly accepted and no observed misses. The implementation uses configured inspection algorithms, including binarization and pattern matching, illustrating a practical alternative to replacing every visual check with a learned model.

118. From Lab to Factory: Pitfalls and Guidelines for Self-/Unsupervised Defect Detection on Low-Quality Industrial Images

     The authors study surface defects on forged steel coupling links photographed from multiple angles. Their initial CS-Flow application produced image-level AUROC around 0.87, yet inspection of anomaly maps showed high scores on backgrounds and regions inconsistent with actual defects. Removing backgrounds did not resolve every failure. Restricting evaluation to similar viewing angles improved results but also simplified the tested problem, which the authors explicitly distinguish from improving performance on the original workload.

119. Dynamic World, Near real-time global 10 m land use land cover mapping

     Brown and colleagues' 2022 Dynamic World system produces land-cover probability maps from individual Sentinel-2 satellite acquisitions. A convolutional model uses spatial and spectral information to distinguish nine surface categories, including trees, crops, water and built areas, on a 10-metre grid. Per-acquisition outputs support examining changes and composing maps for selected periods instead of relying solely on an annual product. The paper documents annotation, cloud masking and held-out assessment, including confusion among vegetation categories.

120. Good practices for estimating area and assessing accuracy of land change

     Olofsson and colleagues's 2014 guidance distinguishes counting classified map pixels from estimating the actual area of a land-cover category. Classification errors can bias the former. A probability sample assessed with higher-quality reference information supports error-adjusted area estimates and uncertainty intervals. When sampling is stratified by mapped class, each stratum's observed reference proportions must be weighted by its share of mapped area; raw sample counts alone generally give the wrong population proportions.

121. Moondream: how does a tiny vision model slap so hard?

     Natural-language task specification can make vision features practical for developers who would otherwise avoid custom model training.

122. Vision AI in 2025 — Peter Robicheaux, Roboflow

     Motion-dependent decisions require repeated frame processing, making latency a reason to place computation near the camera.

123. Google ML Crash Course: Overfitting

     A model can fit training examples yet fail on new data. Training and validation loss curves can expose overfitting, while a test set is useful only when it represents deployment. For vision, new cameras, locations and acquisition conditions are possible distribution changes; fitting and judging parameters require separate data.

124. Vision AI in 2025 — Peter Robicheaux, Roboflow

     The presented DINOv2 feature visualization reportedly exposes object parts and analogous parts across species.

125. Inference Optimal VLMs Need Fewer Visual Tokens and More Parameters

     The scaling experiments use CLIP ViT-L/14 throughout, a common training-data recipe, Qwen-1.5 language-model sizes, and visual-token counts from 576 down to one. Compression uses a TokenPacker variant with convolutional downsampling. These are trained VLM configurations, not a sweep of input resolution through one unchanged encoder-connector-language-model checkpoint. The main response variable averages normalized errors across nine benchmarks, with MME cognition and perception combined. The paper separately examines OCR-oriented tasks and reports that reducing visual tokens harms text-recognition performance even when larger language models compensate for compute. This supports task-dependent compression tradeoffs, but does not supply the requested fixed-connector experiment with separate OCR, counting, and spatial-relation outcomes.

126. Vision AI in 2025 — Peter Robicheaux, Roboflow

     The speaker reports a failure to benefit jointly from class names, annotator instructions, and few-shot visual examples.

127. Moondream: how does a tiny vision model slap so hard?

     Naive caption expansion can preserve incorrect annotations and add unsupported detail; source preprocessing is a substantial part of the training pipeline.

128. State of the Union: Why Local, Why Now

     Use broad models to curate a task-specific dataset, then deploy a specialized model where real-time inference is needed.

129. How Transformers Finally Ate Vision

     With fixed-size image patches, global attention has fourth-power scaling in image side length for its pairwise attention computation.

130. Vision AI in 2025 — Peter Robicheaux, Roboflow

     Image-caption matching can leave models insensitive to details that captions do not distinguish.

131. Building Closed-Loop Evals for a Multimodal Agent at Uber Scale

     Use production-label mismatches to propose configuration changes, but benchmark those changes before registering a new production version.