Linux v4l2 subsystem – definition and how to use

Linux v4l2 subsystem is an application layer access video device

1. V4L2

is a system that provides a lot of access to the hardware video interface, and applications can access it through system calls and video equipment. But because the video equipment is very different, a small number of device drivers can support all interface functions, so before using this system or device, you need to understand what functions this giant device driver supports.

2. Access Process .

2.1. Open device file :

Like other devices, a video device can be regarded as a file , So use open Open file . It can be blocking open , It can also be non blocking open , Non blocking open , If there is no data , It will return an error .

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
……
int camera_fd;
camera_fd = open(“/dev/video0”, O_RDWR); // Blocking open
……
camera_fd = open(“/dev/video0”, O_RDWR | O_NONBLOCK); // Non blocking open
……

2.2. Get the functions supported by the device :

In the use of Video Before equipment , Need to know Video What functions does the device support , If it is an image capture device or an image output device 、 What kind of modulation can be performed on the video signal . Support VBI、 Whether it has audio function, etc . It can be done by VIDIOC_QUERYCAP Command acquisition Video Functions supported by the device . Finally pass the inspection struct v4l2_capability Medium capabilities Variable to obtain the functions supported by the device .

#include <sys/ioctl.h>
#include <linux/videodev2.h>
……
struct v4l2_capability cap = {
0};
int ret = ioctl(camera_fd, VIDIOC_QUERYCAP, &capability);
……
// Determine whether certain functions are supported
if(cap.capabilities & V4L2_CAP_VIDEO_CAPTURE)
printf(“v4l2 device support video capture\n”);
if(cap.capabilities & V4L2_CAP_VIDEO_OUTPUT)
printf(“v4l2 device support video output\n”);
……
The functions of the device are saved in struct v4l2_capability In the structure ,capabilities Variables specifically represent the functions of the device , Functions are defined by macros V4L2_CAP_XXXX Express .

// describe V4L2 The function of the device , Corresponding ioctl command VIDIOC_QUERYCAP
struct v4l2_capability {

__u8 driver[16]; // Name of driver module , Such as “bttv”
__u8 card[32]; // The brand name , Such as “Hauppauge WinTV”
__u8 bus_info[32]; // Bus name , Such as “PCI:” + pci_name(pci_dev)
__u32 version; // Version information ,KERNEL_VERSION
__u32 capabilities; // The overall function of the equipment
__u32 device_caps;
__u32 reserved[3];
};
// capabilities Macro definition of field value
#define V4L2_CAP_VIDEO_CAPTURE 0x00000001 // Image capture device
#define V4L2_CAP_VIDEO_OUTPUT 0x00000002 // Image output device
#define V4L2_CAP_VIDEO_OVERLAY 0x00000004 // Support preview function
#define V4L2_CAP_VBI_CAPTURE 0x00000010 /* Is a raw VBI capture device */
#define V4L2_CAP_VBI_OUTPUT 0x00000020 /* Is a raw VBI output device */
#define V4L2_CAP_SLICED_VBI_CAPTURE 0x00000040 /* Is a sliced VBI capture device */
#define V4L2_CAP_SLICED_VBI_OUTPUT 0x00000080 /* Is a sliced VBI output device */
#define V4L2_CAP_RDS_CAPTURE 0x00000100 /* RDS data capture */
#define V4L2_CAP_VIDEO_OUTPUT_OVERLAY 0x00000200 /* Can do video output overlay */
#define V4L2_CAP_HW_FREQ_SEEK 0x00000400 /* Can do hardware frequency seek */
#define V4L2_CAP_RDS_OUTPUT 0x00000800 /* Is an RDS encoder */
/* Is a video capture device that supports multiplanar formats */
#define V4L2_CAP_VIDEO_CAPTURE_MPLANE 0x00001000
/* Is a video output device that supports multiplanar formats */
#define V4L2_CAP_VIDEO_OUTPUT_MPLANE 0x00002000
/* Is a video mem-to-mem device that supports multiplanar formats */
#define V4L2_CAP_VIDEO_M2M_MPLANE 0x00004000
#define V4L2_CAP_VIDEO_M2M 0x00008000 /* Is a video mem-to-mem device */
#define V4L2_CAP_TUNER 0x00010000 // There is a tuner
#define V4L2_CAP_AUDIO 0x00020000 // Support audio
#define V4L2_CAP_RADIO 0x00040000 /* is a radio device */
#define V4L2_CAP_MODULATOR 0x00080000 /* has a modulator */
#define V4L2_CAP_SDR_CAPTURE 0x00100000 /* Is a SDR capture device */
#define V4L2_CAP_EXT_PIX_FORMAT 0x00200000 /* Supports the extended pixel format */
#define V4L2_CAP_READWRITE 0x01000000 // Support read、write system call
#define V4L2_CAP_ASYNCIO 0x02000000 // Asynchronous Support I/O
#define V4L2_CAP_STREAMING 0x04000000 // Support streaming I/O ioctl function
#define V4L2_CAP_DEVICE_CAPS 0x80000000 /* sets device capabilities field */

2.3. Select device input :

Video The device may have multiple inputs . Like on some chips , The camera controller can be connected to multiple cameras , Which camera needs to be selected as the input source . If there is only one input , There is no need to select .VIDIOC_ENUMINPUT The command can list the information of the corresponding number input device , Information stored in struct v4l2_input In the structure .VIDIOC_S_INPUT You can specify the current input device by number .

#include <sys/ioctl.h>
#include <linux/videodev2.h>
……
struct v4l2_input input = {
0};
// enumeration Video All inputs to the device
while (!ioctl(camera_fd, VIDIOC_ENUMINPUT, &input)) {

printf(“video input name: %s\n”, input.name);
++input.index;
}
……
input.index = XX; // Specify the number entered as XX
ret = ioctl(camera_fd, VIDIOC_S_INPUT, &input); // Set the number to XX The input of is the current input device
……
The input information of the device is saved in struct v4l2_input In the structure .

struct v4l2_input {

__u32 index; /* Which input, Used to determine the input device */
__u8 name[32]; /* Label */
__u32 type; /* Type of input */
__u32 audioset; /* Associated audios (bitfield) */
__u32 tuner; /* enum v4l2_tuner_type */
v4l2_std_id std;
__u32 status;
__u32 capabilities;
__u32 reserved[3];
};
/* Values for the ‘type’ field */
#define V4L2_INPUT_TYPE_TUNER 1
#define V4L2_INPUT_TYPE_CAMERA 2

/* field ‘status’ – general */
#define V4L2_IN_ST_NO_POWER 0x00000001 /* Attached device is off */
#define V4L2_IN_ST_NO_SIGNAL 0x00000002
#define V4L2_IN_ST_NO_COLOR 0x00000004

/* field ‘status’ – sensor orientation */
/* If sensor is mounted upside down set both bits */
#define V4L2_IN_ST_HFLIP 0x00000010 /* Frames are flipped horizontally */
#define V4L2_IN_ST_VFLIP 0x00000020 /* Frames are flipped vertically */

/* field ‘status’ – analog */
#define V4L2_IN_ST_NO_H_LOCK 0x00000100 /* No horizontal sync lock */
#define V4L2_IN_ST_COLOR_KILL 0x00000200 /* Color killer is active */

/* field ‘status’ – digital */
#define V4L2_IN_ST_NO_SYNC 0x00010000 /* No synchronization lock */
#define V4L2_IN_ST_NO_EQU 0x00020000 /* No equalizer lock */
#define V4L2_IN_ST_NO_CARRIER 0x00040000 /* Carrier recovery failed */

/* field ‘status’ – VCR and set-top box */
#define V4L2_IN_ST_MACROVISION 0x01000000 /* Macrovision detected */
#define V4L2_IN_ST_NO_ACCESS 0x02000000 /* Conditional access denied */
#define V4L2_IN_ST_VTR 0x04000000 /* VTR time constant */

/* capabilities flags */
#define V4L2_IN_CAP_DV_TIMINGS 0x00000002 /* Supports S_DV_TIMINGS */
#define V4L2_IN_CAP_CUSTOM_TIMINGS V4L2_IN_CAP_DV_TIMINGS /* For compatibility */
#define V4L2_IN_CAP_STD 0x00000004 /* Supports S_STD */
#define V4L2_IN_CAP_NATIVE_SIZE 0x00000008 /* Supports setting native size */

2.4. Gets and sets the pixel format :

Some cameras support multiple pixel formats , Some cameras only support one pixel format . Therefore, before setting the pixel format, you need to know the pixel format supported by the camera , Then set it .VIDIOC_ENUM_FMT Command enumerates the pixel formats supported by the device ,VIDIOC_S_FMT Command to set the current pixel format of the device .

#include <sys/ioctl.h>
#include <linux/videodev2.h>
struct v4l2_fmtdesc fmtdesc = {
0};
……
// Gets the supported pixel format
while (!ioctl(camera_fd, VIDIOC_ENUM_FMT, &fmtdesc)) {

printf(“fmt: %s\n”, fmtdesc.description);
fmtdesc.index++;
}
……
// Set pixel format
struct v4l2_format fmt = {
0};
fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; // Video capture mode
v4l2_fmt.fmt.pix.width = 720; // Video width
v4l2_fmt.fmt.pix.height = 576; // Video height
v4l2_fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_YUYV; // YUYV Pixel format
v4l2_fmt.fmt.pix.field = V4L2_FIELD_ANY;
ret = ioctl(camera_fd, VIDIOC_S_FMT, &fmt);
……
Here are the structure and enumeration definitions used .

// Used to get the pixel format
struct v4l2_fmtdesc {

__u32 index; /* Format number */
__u32 type; /* enum v4l2_buf_type */
__u32 flags;
__u8 description[32]; /* A string describing the pixel format */
__u32 pixelformat; /* Format fourcc */
__u32 reserved[4];
};
// Used to format pixels
struct v4l2_format {

__u32 type; // Data flow type , from enum v4l2_buf_type Definition
union {

struct v4l2_pix_format pix; /* V4L2_BUF_TYPE_VIDEO_CAPTURE */
struct v4l2_pix_format_mplane pix_mp; /* V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE */
struct v4l2_window win; /* V4L2_BUF_TYPE_VIDEO_OVERLAY */
struct v4l2_vbi_format vbi; /* V4L2_BUF_TYPE_VBI_CAPTURE */
struct v4l2_sliced_vbi_format sliced; /* V4L2_BUF_TYPE_SLICED_VBI_CAPTURE */
struct v4l2_sdr_format sdr; /* V4L2_BUF_TYPE_SDR_CAPTURE */
__u8 raw_data[200]; /* user-defined */
} fmt;
};
enum v4l2_buf_type {

V4L2_BUF_TYPE_VIDEO_CAPTURE = 1,
V4L2_BUF_TYPE_VIDEO_OUTPUT = 2,
V4L2_BUF_TYPE_VIDEO_OVERLAY = 3,
V4L2_BUF_TYPE_VBI_CAPTURE = 4,
V4L2_BUF_TYPE_VBI_OUTPUT = 5,
V4L2_BUF_TYPE_SLICED_VBI_CAPTURE = 6,
V4L2_BUF_TYPE_SLICED_VBI_OUTPUT = 7,
……
};
struct v4l2_pix_format {

__u32 width; // Width
__u32 height; // Height
__u32 pixelformat; // Pixel format
__u32 field; /* enum v4l2_field */
__u32 bytesperline; /* for padding, zero if unused */
__u32 sizeimage;
__u32 colorspace; /* enum v4l2_colorspace */
__u32 priv; /* private data, depends on pixelformat */
__u32 flags; /* format flags (V4L2_PIX_FMT_FLAG_*) */
__u32 ycbcr_enc; /* enum v4l2_ycbcr_encoding */
__u32 quantization; /* enum v4l2_quantization */
};

enum v4l2_field {

V4L2_FIELD_ANY = 0, /* driver can choose from none, top, bottom, interlaced depending on whatever it thinks is approximate … */
V4L2_FIELD_NONE = 1, /* this device has no fields … */
V4L2_FIELD_TOP = 2, /* top field only */
V4L2_FIELD_BOTTOM = 3, /* bottom field only */
V4L2_FIELD_INTERLACED = 4, /* both fields interlaced */
V4L2_FIELD_SEQ_TB = 5, /* both fields sequential into one buffer, top-bottom order */
V4L2_FIELD_SEQ_BT = 6, /* same as above + bottom-top order */
V4L2_FIELD_ALTERNATE = 7, /* both fields alternating into separate buffers */
V4L2_FIELD_INTERLACED_TB = 8, /* both fields interlaced, top field first and the top field is transmitted first */
V4L2_FIELD_INTERLACED_BT = 9, /* both fields interlaced, top field first and the bottom field is transmitted first */
};

2.5. Request buffer :

Video The video data captured by the device should be stored in a pre allocated buffer . Use VIDIOC_REQBUFS Command to request a buffer from the kernel . Before you apply , You need to set the number of requested buffers nr_bufs、 Buffer data stream type type And buffer memory usage memory.

#include <sys/ioctl.h>
#include <linux/videodev2.h>
struct v4l2_requestbuffers req = {
0};
req.count = nr_bufs; // Number of caches
req.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
req.memory = V4L2_MEMORY_MMAP; // How memory is mapped , Can improve efficiency , Reduce memory usage
……
ret = ioctl(camera_fd, VIDIOC_REQBUFS, &req);
……
Here is struct v4l2_requestbuffers Specific definition of structure .

struct v4l2_requestbuffers {

__u32 count; // Number of buffers
__u32 type; // Data flow type , Usually it is V4L2_BUF_TYPE_VIDEO_CAPTURE
__u32 memory; // Buffer memory usage , Usually it is V4L2_MEMORY_MMAP
__u32 reserved[2];
};
enum v4l2_memory {

V4L2_MEMORY_MMAP = 1,
V4L2_MEMORY_USERPTR = 2,
V4L2_MEMORY_OVERLAY = 3,
V4L2_MEMORY_DMABUF = 4,
};
2.6. Buffer mapping
Use read This will cause data to be copied back and forth between user space and kernel space , Low efficiency , And it takes up memory in both user space and kernel space , Spending big . Usually, the buffer uses memory mapping ,mmap Put the driver videobuf2 Managed memory mapped to user space , Applications have direct access to videobuf2 Managed memory , No data copying occurs , It’s efficient , Low memory usage .VIDIOC_QUERYBUF Command to get buffer information ,VIDIOC_QBUF Add the buffer to the kernel queue .

#include <stddef.h>
#include <string.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <linux/videodev2.h>

struct video_buf {

void* start;
size_t length;
}
struct v4l2_buffer v4l2_buf;
struct video_buf* buf = calloc(req.count, sizeof(struct video_buf));
……
for (i = 0; i < req.count; i++) {
// Map all the requested buffers
memset(&v4l2_buf, 0, sizeof(v4l2_buf));
buf.index = i;
// Get the number i Buffer information for
ret = ioctl(camera_fd, VIDIOC_QUERYBUF, &v4l2_buffer);
……
buf[i].length = v4l2_buf.length; // Record buffer length
// Start mapping , When the mapping is complete , The application obtains video data through this address
buf[i].start = mmap(NULL, v4l2_buf.length, PORT_READ | PORT_WRITE,
MAP_SHARED, camera_fd, v4l2_buf.m.offset);
……
// Add the buffer to the kernel’s buffer queue
ret = ioctl(camera_fd, VIDIOC_QBUF, v4l2_buf);
……
}

2.6. Buffer mapping :

Use read This will cause data to be copied back and forth between user space and kernel space , Low efficiency , And it takes up memory in both user space and kernel space , Spending big . Usually, the buffer uses memory mapping ,mmap Put the driver videobuf2 Managed memory mapped to user space , Applications have direct access to videobuf2 Managed memory , No data copying occurs , It’s efficient , Low memory usage .VIDIOC_QUERYBUF Command to get buffer information ,VIDIOC_QBUF Add the buffer to the kernel queue .

#include <stddef.h>
#include <string.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <linux/videodev2.h>

struct video_buf {

struct v4l2_buffer The detailed definition of is as follows .

struct v4l2_buffer {

__u32 index;
__u32 type;
__u32 bytesused;
__u32 flags;
__u32 field;
struct timeval timestamp;
struct v4l2_timecode timecode;
__u32 sequence;

/* memory location */
__u32 memory;
union {

__u32 offset;
unsigned long userptr;
struct v4l2_plane *planes;
__s32 fd;
} m;
__u32 length;
__u32 reserved2;
__u32 reserved;
};

2.7. Start capturing video :

After the previous preparations , You can now enable the device , Start collecting video data .VIDIOC_STREAMON Command enable device , Start capturing video .

#include <sys/ioctl.h>
#include <linux/videodev2.h>
enum v4l2_buf_type type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
……
ret = ioctl(camera_fd, VIDIOC_STREAMON, &type);
……

2.8. Processing video data :

Buffers are organized in the form of ring queues in the kernel , When processing data, get a buffer from the ring queue , Finished processing , Put the buffer into the ring queue . Application through VIDIOC_DQBUF Command to get a buffer from the queue , Use VIDIOC_QBUF Add the buffer to the queue . There is no copy of data in the whole process , Than traditional read and write I/O The method is much more efficient , Memory usage is also greatly reduced .

#include <sys/ioctl.h>
#include <linux/videodev2.h>
void* data = NULL;
size_t length = 0;
struct v4l2_buffer v4l2_buf = {
0};
for (;;) {

ret = ioctl(camera_fd, VIDIOC_QBUF, &v4l2_buf); // Get a buffer from the ring queue
……
data = buf[v4l2_buf.index].start; // Buffer address
length = buf[v4l2_buf.index].length // Buffer data length
……
ret = ioctl(camera_fd, VIDIOC_QBUF, &v4l2_buf); // Put the buffer into the ring queue
……
}

2.9. Stop capturing video :

The application can use VIDIOC_STREAMOFF Command to stop video capture , At the same time, the equipment is prohibited .

#include <sys/ioctl.h>
#include <linux/videodev2.h>
……
enum v4l2_buf_type type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
ret = ioctl(camera_fd, VIDIOC_STREAMOFF, &type);
……

2.10. Turn off the device ;

When the device is no longer in use , You can use close Turn off settings ,close After the call, the previously requested buffer will be released .

Linux v4l2 subsystem – definition and how to use

1. V4L2