Wrong chroma conversion from RV24 to I420 (swscale)

Opening a directshow device with RV24 chroma (select it from dshow panel with :dshow-config) opens a video with scrabled colors. I suspect the GetPixels method @ modules/video_filter/swscale.c not taking into account

assigned to @dionoea

added Component::Video: filters label

Hello, i started hacking on this but had to stall because win32 directx video output seems broken (and i can no more test) (and also hacking on dshow so not much time).

Nevertheless i suspect incorect endianess mappings on modules/codec/avcodec/chroma.c

I d make #if defined(WORDS_BIGENDIAN)

define VLC_RGB_ES( fcc, leid, beid ) \

{ fcc, beid, 0, 0, 0 },

#else

define VLC_RGB_ES( fcc, leid, beid ) \

{ fcc, leid, 0, 0, 0 },

#endif

#define VLC_RGB( fcc, leid, beid, rmask, gmask, bmask ) \

• { fcc, leid, rmask, gmask, bmask }, \
• { fcc, beid, bmask, gmask, rmask }, \

• { fcc, leid, bmask, gmask, rmask }, \
• { fcc, beid, rmask, gmask, bmask }, \

And later on

• VLC_RGB( VLC_FOURCC('R','G','B','4'), PIX_FMT_RGB4, PIX_FMT_BGR4, 0x10, 0x06, 0x01 )
• VLC_RGB( VLC_FOURCC('R','G','B','8'), PIX_FMT_RGB8, PIX_FMT_BGR8, 0xC0, 0x38, 0x07 )

• VLC_RGB( VLC_FOURCC('R','G','B','4'), PIX_FMT_BGR4, PIX_FMT_RGB4, 0x10, 0x06, 0x01 )
• VLC_RGB( VLC_FOURCC('R','G','B','8'), PIX_FMT_BGR8, PIX_FMT_RGB8, 0xC0, 0x38, 0x07 )

(but don;t have test streams on these)

(i have test stream on the RV24 though)

Also when on little endian do we need the big endian mapping ?

I d make

#if defined(WORDS_BIGENDIAN) 
# define VLC_RGB_ES( fcc, leid, beid ) \\
    { fcc, beid, 0, 0, 0 },

#else 
# define VLC_RGB_ES( fcc, leid, beid ) \\
    { fcc, leid, 0, 0, 0 },
#endif

#define VLC_RGB( fcc, leid, beid, rmask, gmask, bmask ) \\
 - { fcc, leid, rmask, gmask, bmask }, \\ 
 - { fcc, beid, bmask, gmask, rmask }, \\ 
 + { fcc, leid, bmask, gmask, rmask }, \\ 
 + { fcc, beid, rmask, gmask, bmask }, \\

And later on

- VLC_RGB( VLC_FOURCC('R','G','B','4'), PIX_FMT_RGB4, PIX_FMT_BGR4, 0x10, 0x06, 0x01 ) 
- VLC_RGB( VLC_FOURCC('R','G','B','8'), PIX_FMT_RGB8, PIX_FMT_BGR8, 0xC0, 0x38, 0x07 ) 
+ VLC_RGB( VLC_FOURCC('R','G','B','4'), PIX_FMT_BGR4, PIX_FMT_RGB4, 0x10, 0x06, 0x01 ) 
+ VLC_RGB( VLC_FOURCC('R','G','B','8'), PIX_FMT_BGR8, PIX_FMT_RGB8, 0xC0, 0x38, 0x07 )

COrrect format

This ought to be fixed in the vout rework, unless it is a bug in the DirectShow input.

If the bug is still present please update the bug or file a new one.

added Status::invalid label

closed