Using AVKit to detect luminosity

Question

I am working on an app using SwiftUi that leverages the device camera to detect luminosity as described in top answer of this post. The captureOutput(_:didOutput:from:) function in the top answer was used to calculate luminosity. According to Apple Docs this function is intended to notify a delegate that a new video frame was written, and so I have placed this function in a VideoDelegate class. This delegate is then set in a VideoStream class that handles the logic of asking permissions and setting up an AVCaptureSession. My question is how to access the luminosity value calculated within the delegate inside my SwiftUI view?

struct ContentView: View {
    @StateObject var videoStream = VideoStream()
    
    var body: some View {
        Text("\(videoStream.luminosityReading) ?? Detecting...")
            .padding()
    }
}

class VideoStream: ObservableObject {
    
    @Published var luminosityReading : Double = 0.0 // TODO get luminosity from VideoDelegate
    var session : AVCaptureSession!
    
    init() {
        authorizeCapture()
    }
    
    func authorizeCapture() {
        // permission logic and call to beginCapture()
    }
    
    func beginCapture() {        
        session = AVCaptureSession()
        session.beginConfiguration()
        let videoDevice = bestDevice() // func definition omitted for readability 
        guard
            let videoDeviceInput = try? AVCaptureDeviceInput(device: videoDevice),
            session.canAddInput(videoDeviceInput)
        else {
            print("Camera selection failed")
            return
        }
        
        let videoOutput = AVCaptureVideoDataOutput()
        guard
            session.canAddOutput(videoOutput)
        else {
            print("Error creating video output")
            return
        }
        
        session.sessionPreset = .high
        session.addOutput(videoOutput)
        
        let queue = DispatchQueue(label: "VideoFrameQueue")
        let delegate = VideoDelegate()
        videoOutput.setSampleBufferDelegate(delegate, queue: queue)
        
        session.commitConfiguration()
        session.startRunning()
    }
}

class VideoDelegate: NSObject, AVCaptureVideoDataOutputSampleBufferDelegate {
    func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
        //Retrieving EXIF data of camara frame buffer
        let rawMetadata = CMCopyDictionaryOfAttachments(allocator: nil, target: sampleBuffer, attachmentMode: CMAttachmentMode(kCMAttachmentMode_ShouldPropagate))
        let metadata = CFDictionaryCreateMutableCopy(nil, 0, rawMetadata) as NSMutableDictionary
        let exifData = metadata.value(forKey: "{Exif}") as? NSMutableDictionary
        
        let FNumber : Double = exifData?["FNumber"] as! Double
        let ExposureTime : Double = exifData?["ExposureTime"] as! Double
        let ISOSpeedRatingsArray = exifData!["ISOSpeedRatings"] as? NSArray
        let ISOSpeedRatings : Double = ISOSpeedRatingsArray![0] as! Double
        let CalibrationConstant : Double = 50
        
        //Calculating the luminosity
        let luminosity : Double = (CalibrationConstant * FNumber * FNumber ) / ( ExposureTime * ISOSpeedRatings )
         
        // how to pass value of luminosity to `VideoStream`? 
    }
}

Does your `VideoDelegate` *have to* be a separate class? Why not make `VideoStream` conform to `AVCaptureVideoDataOutputSampleBufferDelegate` and implement the delegate method there? — jnpdx, Sep 04 '22 at 22:59
This is a great solution, I've gotten the app running but unfortunately `luminosityReading` is not changing. — John Harrington, Sep 05 '22 at 01:10
Did you set it anywhere? Did you use the debugger to see what is getting called? — jnpdx, Sep 05 '22 at 01:29
Yes I've now set `delegate = self` and added a print statement to `captureOutput` and have determined it is never being called. Updated code is at https://github.com/harr1424/LibreLightSensor/blob/main/LibreLightSensor/ContentView.swift. this seems to be a common problem with a known solution force unwrapping the parameters for `captureOutput` but this also has not resulted in the function being called. — John Harrington, Sep 05 '22 at 01:35
https://stackoverflow.com/questions/44999250/captureoutput-not-being-called — John Harrington, Sep 05 '22 at 01:36

score 1 · Accepted Answer · answered Sep 05 '22 at 01:09

1

As discussed in the comments, the lowest friction option would be to have VideoStream conform to AVCaptureVideoDataOutputSampleBufferDelegate and implement the delegate method there.

answered Sep 05 '22 at 01:09

jnpdx

45,847
6
64
94

Using AVKit to detect luminosity

1 Answers1