## GstVideoContextInterface

GstVideoContextInterface — Interface to handle video library context

## Synopsis

    #define             GST_IS_VIDEO_CONTEXT                (obj)
                        GstVideoContext;
    struct              GstVideoContextInterface;
    GType               gst_video_context_iface_get_type    (void);
    void                gst_video_context_set_context       (GstVideoContext *context,
                                                             const gchar *type,
                                                             const GValue *value);
    void                gst_video_context_set_context_string
                                                            (GstVideoContext *context,
                                                             const gchar *type,
                                                             const gchar *value);
    void                gst_video_context_set_context_pointer
                                                            (GstVideoContext *context,
                                                             const gchar *type,
                                                             gpointer value);
    void                gst_video_context_set_context_object
                                                            (GstVideoContext *context,
                                                             const gchar *type,
                                                             GObject *value);
    void                gst_video_context_prepare           (GstVideoContext *context,
                                                             const gchar **types);
    gboolean            gst_video_context_message_parse_prepare
                                                            (GstMessage *message,
                                                             const gchar ***types,
                                                             GstVideoContext **ctx);
    GstQuery *          gst_video_context_query_new         (const gchar **types);
    gboolean            gst_video_context_run_query         (GstElement *element,
                                                             GstQuery *query);
    const gchar **      gst_video_context_query_get_supported_types
                                                            (GstQuery *query);
    void                gst_video_context_query_parse_value (GstQuery *query,
                                                             const gchar **type,
                                                             const GValue **value);
    void                gst_video_context_query_set_value   (GstQuery *query,
                                                             const gchar *type,
                                                             GValue *value);
    void                gst_video_context_query_set_string  (GstQuery *query,
                                                             const gchar *type,
                                                             const gchar *value);
    void                gst_video_context_query_set_pointer (GstQuery *query,
                                                             const gchar *type,
                                                             gpointer value);
    void                gst_video_context_query_set_object  (GstQuery *query,
                                                             const gchar *type,
                                                             GObject *value);

## Description

The Video Context interface enable sharing video context (such as display name, X11 Display, VA-API Display, etc) between neighboor elements and the application.

### Note

The GstVideoContext interface is unstable API and may change in future. One can define GST_USE_UNSTABLE_API to acknowledge and avoid this warning.

### For Element

This interface shall be implement by group of elements that need to share a specific video context (like VDPAU, LibVA, OpenGL elements) or by video sink in order to let the application select appropriate display information (like the X11 display name) when those sink are auto-plugged.

Along with implementing the interface, elements will need to query neighboor elements or send message to the application when preparing the context (see `gst_video_context_prepare()`). They also need to reply to the neighboors element queries, so the context can be shared without the application help.

Elements that are guarantied to have both upstream and downstream neighboors element implementing the GstVideoContext (like the gloverlay element in gst-plugins-opengl) is not required to also implement the interface. Relying on neighboors query shall be sufficient (see `gst_video_context_run_query()`).

The query is an application query with a structure name set to "prepare-video-context" and an array of supported video context types set in the field named "types". This query shall be send downstream and upstream, iterating the pads in order to find neighboors regardless of a static (sink to src) or a dynamic (src to sink) activation. Element should used the helpers method `gst_video_context_prepare()` (or `gst_video_context_run_query()` if no GstVideoContext interface) to correctly execute the query . The result is set using the query helper functions, the structures fields name being "video-context-type" as string and "video-context" as a GValue.

If the query is not handled by any neighboor, the element should ask the application using the "prepare-video-context" message. The application may then use the interface to set the video context information. If no context was set, the element shall create one using default configuration. Elements with multiple src or sink pad shall implement proper locking to prevent the race of parallel queries being replied.

Well known video-context are: "x11-display-name" a string representing the X11 display to use, "x11-display" the X11 Display structure, "va-display", the VADisplay structure and more.

### For Application

In the case there is no neighboor element with video context to share, the element will first turn toward the application, by sending a "prepare-video-context" message. This message is sent along with a list of supported display types. The application can optionally reply to this message by calling appropriate setter through the GstVideoContext interface. If the application supports more then one video context type, it should choose the first one to occure in the supported list. It's important to remember that the message is delivered from the streaming thread, and appropriate locking should be considered. If the application does not have a video context to share, the element will simply allocate one base on default settings. Usually, only applications using OpenGL base sink, or running on special X11 display need to share a video context.

### Note

Applications sharing X11 Display structure should always initialize the X11 threading support using `XInitThreads()` as GStreamer will need to manipulate the display from a separeate threads.

### Example using ClutterVideoGstVideoSink

This example is for user of ClutterGstVideoSink element, the ClutterGstPlayer object transparently handle this.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51

    #if CLUTTER_WINDOWING_X11
    static GstBusSyncReply
    on_sync_message (GstBus * bus, GstMessage * message, gpointer user_data)
    {
      Display *display = user_data;
      GstVideoContext *context;
      const gchar **types;

      if (gst_video_context_message_parse_prepare (message, &types, &context)) {
        gint i;

        for (i = 0; types[i]; i++) {

          if (!strcmp(types[i], "x11-display")) {
            gst_video_context_set_context_pointer (context, "x11-display", display);
          }
          else if (!strcmp(types[i], "x11-display-name")) {
            gst_video_context_set_context_string (context, "x11-display-name",
                DisplayString (display));
          } else {
            continue;
          }

          gst_message_unref (message);
          return GST_BUS_DROP;
        }
      }

      return GST_BUS_PASS;
    }
    #endif

    gint
    main (gint argc, gchar **argv)
    {
      GstBin *pipeline;
      GstBus *bus;

      ...

      bus = gst_pipeline_get_bus (GST_PIPELINE (pipeline));

      #if CLUTTER_WINDOWING_X11
      gst_bus_set_sync_handler (priv->bus, on_sync_message,
          clutter_x11_get_default_display ());
      #endif

      gst_object_unref (GST_OBJECT (priv->bus));

      ...
    }

## Details

### GST_IS_VIDEO_CONTEXT()

    #define GST_IS_VIDEO_CONTEXT(obj)           (GST_IMPLEMENTS_INTERFACE_CHECK_INSTANCE_TYPE ((obj), GST_TYPE_VIDEO_CONTEXT))

### GstVideoContext

    typedef struct _GstVideoContext GstVideoContext;

Opaque GstVideoContext data structure.

### struct GstVideoContextInterface

    struct GstVideoContextInterface {
      GTypeInterface parent;

      /* virtual functions */
      void (*set_context) (GstVideoContext * context,
                           const gchar * type,
                           const GValue * value);
    };

GstVideoContextInterface interface.

GTypeInterface `parent`; | parent interface type.
`set_context` () | vmethod to set video context.

### gst_video_context_iface_get_type ()

    GType               gst_video_context_iface_get_type    (void);

### gst_video_context_set_context ()

    void                gst_video_context_set_context       (GstVideoContext *context,
                                                             const gchar *type,
                                                             const GValue *value);

This is a wrapper for the `set_context()` virtual method. It is suggested to use one of the helpers to avoid having to manipulate GValue

`context` : | an element implementing GstVideoContext
`type` : | the type of display being set
`value` : | a GValue containing the context

### gst_video_context_set_context_string ()

    void                gst_video_context_set_context_string
                                                            (GstVideoContext *context,
                                                             const gchar *type,
                                                             const gchar *value);

This helper is commonly used for setting video context represented by a string like the X11 display name ("x11-display-name")/

`context` : | an element implementing GstVideoContext
`type` : | the type of display being set
`string` : | a string representing the video context

### gst_video_context_set_context_pointer ()

    void                gst_video_context_set_context_pointer
                                                            (GstVideoContext *context,
                                                             const gchar *type,
                                                             gpointer value);

This helper is used for setting video context using a pointer, typically to a structure like the X11 Display ("x11-display") or the VADisplay ("vaapi-display").

### Note

Users of X11 Display should ensure that `XInitThreads()` was called before opening the display.

`context` : | an element implementing GstVideoContext
`type` : | the type of display being set
`pointer` : | a pointer to the video context

### gst_video_context_set_context_object ()

    void                gst_video_context_set_context_object
                                                            (GstVideoContext *context,
                                                             const gchar *type,
                                                             GObject *value);

This is for video context that are GObject, this helper allow taking benifit of the GObject refcounting. It is particularly handy for element to have refcounting as the order in which element will stop using the display is not defined.

`context` : | an element implementing GstVideoContext
`type` : | the type of display being set
`object` : | a GObject resenting the display

### gst_video_context_prepare ()

    void                gst_video_context_prepare           (GstVideoContext *context,
                                                             const gchar **types);

This method run "prepare-video-context" custom query dowstream, and upstream. If * the query has a reply, it sets the context value using `gst_video_context_set_context()`. Otherwise, it sends a "prepare-video-context" message to the application. The element can then continue video context initialization.

`context` : | an element implementing GstVideoContext interface
`types` : | an array of supported types, prefered first

### gst_video_context_message_parse_prepare ()

    gboolean            gst_video_context_message_parse_prepare
                                                            (GstMessage *message,
                                                             const gchar ***types,
                                                             GstVideoContext **ctx);

This helper shall be used by application to simply handling of the "prepare-video-context" message.

Rerturns: FALSE is the message was not valid "prepare-video-context" element message, otherwise TRUE with `types` and `context` set.

`message` : | a GstMessage
`types` : | return value for supported types
`context` : | return value for the element the implements GstVideoContext

### gst_video_context_query_new ()

    GstQuery *          gst_video_context_query_new         (const gchar **types);

Create a new custom GstQuery with structure name "prepare-video-context".

`types` : | a string array of video context types

### gst_video_context_run_query ()

    gboolean            gst_video_context_run_query         (GstElement *element,
                                                             GstQuery *query);

This helper runs the query on each downstream, then upstream pads in an element. This is called by `gst_video_context_prepare()`. This method is only used directly within elements that are required to have two neighboors elements with appropriate video context. This would be the case of specialized filters that only manipulate non-raw buffers (e.g. gldeinterlace). Those elements do not have to implement GstVideoContext interface.

`element` : | a GstElement
`query` : | a GstQuery

### gst_video_context_query_get_supported_types ()

    const gchar **      gst_video_context_query_get_supported_types
                                                            (GstQuery *query);

`query` : | a GstQuery
Returns : | An array of supported video context types

### gst_video_context_query_parse_value ()

    void                gst_video_context_query_parse_value (GstQuery *query,
                                                             const gchar **type,
                                                             const GValue **value);

Helper to extract the video context type and value from a GstQuery.

`query` : | a GstQuery
`type` : | return video context type
`value` : | return video context GValue

### gst_video_context_query_set_value ()

    void                gst_video_context_query_set_value   (GstQuery *query,
                                                             const gchar *type,
                                                             GValue *value);

Helper to set the video context as a GValue inside the GstQuery.

`query` : | a GstQuery
`type` : | the video context type
`value` : | a GValue set with video context

### gst_video_context_query_set_string ()

    void                gst_video_context_query_set_string  (GstQuery *query,
                                                             const gchar *type,
                                                             const gchar *value);

Helper to set the video context as a string inside the GstQuery.

`query` : | a GstQuery
`type` : | the video context type
`value` : | a string representing the video context

### gst_video_context_query_set_pointer ()

    void                gst_video_context_query_set_pointer (GstQuery *query,
                                                             const gchar *type,
                                                             gpointer value);

Helper to set the video context as a gpointer inside the GstQuery.

`query` : | a GstQuery
`type` : | the video context type
`value` : | a gpointer representing the video context

### gst_video_context_query_set_object ()

    void                gst_video_context_query_set_object  (GstQuery *query,
                                                             const gchar *type,
                                                             GObject *value);

Helper to set the video context as a GObject inside the GstQuery.

`query` : | a GstQuery
`type` : | the video context type
`value` : | a GObject representing the video context