S3C6410 MFC H.264 mp4 player

S3C6410 can do hardware video decode including H.264.
There's an example code from Samsung shows how to hardware decode H.264 raw data, but in the real world application this is not so useful.
Decode & Playing H.264 mp4 file is more reality. So I would like to share the implementation.

I use ffmpeg library to decode mp4 mux in order to get H.264 raw data and then send it to S3C6410 MFC to decode and finally play on the LCD panel.

Requirement
ffmpeg-0.6.1
Samsung FIMV_MFC_V1.0

Source code
https://docs.google.com/open?id=0Bx901QMLr9WlTTR1a0ZKaXVobW8

The most important part is to handle data from

  av_read_frame(formatContext, &pkt);

if you pass pkt.data to decode you got fail to decode. Another filter is required.

  typedef struct H264BSFContext {
    uint8_t  length_size;
    uint8_t  first_idr;
    uint8_t *sps_pps_data;
    uint32_t size;
  } H264BSFContext;

  AVBitStreamFilterContext *filterContext = (AVBitStreamFilterContext *)malloc(sizeof(AVBitStreamFilterContext));
  filterContext->priv_data = (H264BSFContext *)malloc(sizeof(H264BSFContext));
  memset(filterContext->priv_data, 0, sizeof(H264BSFContext));
  filterContext->filter = &h264_mp4toannexb_bsf;
  filterContext->parser = 0;
  filterContext->next = 0;
...
  r = av_read_frame(formatContext, &pkt);
...
    unsigned char *lp = 0;
    int len = 0;
    filterContext->filter->filter(filterContext, codecContext, NULL, &lp, &len, pkt.data, pkt.size, (pkt.flags | PKT_FLAG_KEY)); -->> filter here

    decode_mfc(lp, len);

Ubuntu Intel I5-2500K & Z68 chipset

上次組裝電腦已經是6~7年前的事了,昨天終於下定決心把公司那台半古董桌機換下來,一方面增加工作效率也讓已經逐漸老化的硬碟退休.
之前已經在網路上研究了配備一陣子,主要還是與Ubuntu的相容性要好.最後定案的配備如下

Intel I5-2500K (有K的表示可超頻,貴幾百吧)
ASRock Z68 PRO3 GEN3 (這張Ubuntu直上,還有一個COM port,重點是便宜啊)
Kongston 4GB DDR3-1333 x 2 (共8G memory,這在十年前簡直是無法想像)
Seagate SATA3 1T ST1000DM003
CM Hyper 212 PLUS 塔型散熱器 (CPU不怕著涼的,裝大顆一點就對了)
振華400W power

裝好搬回來後直接裝Ubuntu 11.10,一路順暢都沒有什麼問題.
我最擔心的Intel HD 3000 graphic controller也能正常工作,Perfect

接著是一些監控系統資訊的方式
1.CPU 溫度

modprobe coretemp
sensors

gigijoe@gigijoe-i5-2500k:~$ sensors
coretemp-isa-0000
Adapter: ISA adapter
Physical id 0:  +31.0°C  (high = +80.0°C, crit = +98.0°C)
Core 0:         +25.0°C  (high = +80.0°C, crit = +98.0°C)
Core 1:         +29.0°C  (high = +80.0°C, crit = +98.0°C)
Core 2:         +26.0°C  (high = +80.0°C, crit = +98.0°C)
Core 3:         +30.0°C  (high = +80.0°C, crit = +98.0°C)

2.硬體資訊
sudo lshw

3.CPU 超頻
現在的超頻很容易,直接在BIOS內用懶人設定選取希望的頻率即可.
3.3G -> 4.6G還蠻穩定的,CPU溫度只稍微上升.

在linux環境下cat /pro/cpuinfo會發現超頻之後的CPU clock一樣顯示是3.3G.
嗯,經過一番google後先試著用wine 跑 CPU-z這個Windows上廣泛使用的程式...失敗.
最後正解是一隻叫i7z的程式 http://code.google.com/p/i7z/


Cpu speed from cpuinfo 3292.00Mhz
cpuinfo might be wrong if cpufreq is enabled. To guess correctly try estimating
Linux's inbuilt cpu_khz code emulated now
True Frequency (without accounting Turbo) 3292 MHz
  CPU Multiplier 33x || Bus clock frequency (BCLK) 99.76 MHz

Socket [0] - [physical cores=4, logical cores=4, max online cores ever=4]
  TURBO ENABLED on 4 Cores, Hyper Threading OFF
  True Frequency 3391.76 MHz (99.76 x [34])
  Max TURBO Multiplier (if Enabled) with 1/2/3/4 Cores is  60x/60x/60x/60x
  Current Frequency 1793.89 MHz [99.76 x 17.98] (Max of below)
        Core [core-id]  :Actual Freq (Mult.)      C0%   Halt(C1)%  C3 %   C6 %
        Core 1 [0]:       1793.89 (17.98x)      38.2    38.7     9.4    31.1
        Core 2 [1]:       1655.03 (16.59x)      9.69    13.6    5.07    76.5
        Core 3 [2]:       1657.92 (16.62x)      2.77    2.38     1.3    94.9
        Core 4 [3]:       1618.08 (16.22x)         1    0.747      0    98.9



C0 = Processor running without halting
C1 = Processor running with halts (States >C0 are power saver)
C3 = Cores running with PLL turned off and core cache turned off
C6 = Everything in C3 + core state saved to last level cache
  Above values in table are in percentage over the last 1 sec

3.還有一個小問題是boot會卡在Checking Battery State.
按Ctrl+Alt+F1進 console login.
sudo apt-get install gdm
安裝過程中選lightdm

或是
sudo dpkg-reconfigure gdm

然後reboot就解決囉

Typhoon Race 2011 29.09 record by Steve Gigigjoe with Dingo

Avoid zombie process

在某些應用中常會需要執行其他的程式,這時候就會需要呼叫fork產生child process並在此執行其他程式.
而當被呼叫的程式執行結束child process也就跟著結束,這時child process就成為zombie process.

if(fork() == 0) { /* Child process*/
  /* Run program */
  execl("/bin/ntpdate", "ntpdate", "tick.stdtime.gov.tw", "tock.stdtime.gov.tw", (char *)0);
  exit(0);
  /* zombie process */
}

/* Parent process or error */
 
wait(0);

zombie process存在的意義是等待parent process來處理child process結束後的事情,例如程式的傳回值.

有兩種方式來避免zombie process
1.明確指出parent process不需要知道child process的結束狀態
在parent process 程式碼中加上

signal(SIGCHLD, SIG_IGN); /*  Do not wait for child process */

2.連續fork兩次,藉由結束first child process的方式讓second child process的parent process為pid 1  (通常是 init)

pid_t pid;
if((pid = fork()) == 0)  { /*First child process*/
  if((pid = fork()) > 0)
      exit(0); /* Terminate first child process */
  /* Second child process. Run program */
  execl("/bin/ntpdate", "ntpdate", "tick.stdtime.gov.tw", "tock.stdtime.gov.tw", (char *)0);
  exit(0);
}
 
/* Parent process or error */

Embedded Linux IPv6 ready logo phase-2

The past month I'm working on IPv6 ready test on an embedded linux platform.
Since the kernel is really old 2.6.10 and I have to do some porting from newer kernel to get IPv6 ready test pass. Last week I finally all pass and below are some tips.

1.The test node (TN) is FreeBSD 7.4 release with v6eval-3.3.2 / v6eval-remotes-3.0 / Self_Test_5.0.0

It's very important that to disable ipv6 autoconfiguration on TN, because the testing suit needs clean ipv6 environment.

To disable ipv6 autoconfiguration, edit /etc/rc.conf and set

ipv6_enable="NO"

then after TN boot, the ipv6 link local address (fe80:X) won't be exist.

2./usr/local/v6eval/etc/tn.def

#
# tn.def
#
#  Information about the Tester Node (TN)
#

#
# Remote Controal Configuration
#
RemoteDevice    cuad0
RemoteDebug     0
RemoteIntDebug  0
RemoteLog       1
RemoteSpeed     0
RemoteLogout    0
RemoteMethod    serial
#filter ipv6

#linkname       interface       BOGUS ether source address
#               name            of the Tester Interface
Link0           rl0             00:00:00:00:01:00
#Link1          de1             00:00:00:00:01:01
#Link2          de2             00:00:00:00:01:02
#Link3          de4             00:00:00:00:01:03

3./usr/local/v6eval/etc/nut.def

#
# nut.def
#
#  Information about the Node Under Test (NUT)
#

# System type
System          linux-v6

# System information
TargetName      FreeBSD/i386 4.9-RELEASE + kame-20040726-freebsd49-snap

# Name
HostName        HL200

# Type
#   host, router, special
Type            host

# Super user name and it's password
# if you select manual as "System", you don't care "User" and "Password"
#
User            root
Password        1234

#linkname       interface       The EXACT ether source address
#               name            of the Interface Under Test
Link0           eth0            00:AE:45:BC:84:1E
#Link0          eth0            00:60:37:c0:ff:ee
#Link1          fxp1            00:00:92:a7:6d:f6
#Link2          de0             00:c0:f6:b0:aa:ef
#Link3          de1             00:00:92:a7:6d:f8
#Link4          de2             00:90:27:14:ce:e3

4./usr/local/lib/perl5/site_perl/5.10.1/V6evalRemote.pm
This file is really important for self test. It makes the TN to communicate NUT through serial RS232 port. TN can login / logout NUT system even more reboot NUT.

So, find key words below in this file and get it right

    $Type="linux-v6";
    $Device="cuad0";
    $User="root";
    $Password="1234";

The TN will detect message from NUT to login / logout. For example "login: " and "Password: ". Also get it right for your NUT's Behavior

    # login prompt
    %prompt_user = (
           ...
           'linux-v6',         'login: ',
           ...
    );

    # password prompt
    %prompt_password = (
       ...
        'linux-v6',             'Password: ',
        ...
    );

    # command prompt
    %prompt_command = (
        ...
        'linux-v6',              '(\$|#) ',
        ...
    );

    and so on

5.Test TN <-RS232-> NUT
If you can see login screen or promopt, it works

touch /var/log/aculog
chown uucp:dialer /var/log/aculog
chmod 660 /var/log/aculog
cu -l /dev/cuad0
Connected

[root@HL200 ~]$

6.Remote test
To test if TN can login then logout NUT

/usr/local/v6eval/bin/linux-v6/loginout.rmt -o1

HL200 login: root
Password:


BusyBox v1.2.2 (2011.07.20-06:33+0000) Built-in shell (ash)
Enter 'help' for a list of built-in commands.

[root@HL200 ~]$
[root@HL200 ~]$ exit

7.Run self test

cd Self_Test_5-0-0
make clean
make ipv6ready_p2_host

8.Run parts of self test

There are 5 test sections, you can test separated.

cd nd.p2/
make clean
make ipv6ready_p2_host

Also can test only few items, for example test items 10 to 20

make AROPT="-s 10 -e 20" ipv6ready_p2_host

9.Embedded linux setup.
There are some sysctl for ipv6 need to be set correct.

echo 0 > /proc/sys/net/ipv6/conf/default/forwarding
echo 2 > /proc/sys/net/ipv6/conf/default/dad_transmits
echo 1 > /proc/sys/net/ipv6/conf/default/use_tempaddr
echo 2 > /proc/sys/net/ipv6/conf/default/accept_dad
echo 1 > /proc/sys/net/ipv6/conf/default/optimistic_dad
echo 1 > /proc/sys/net/ipv6/conf/default/accept_redirects

Remember to set promisc and allmulti to ethernet port.
Another tip is to sleep 10 seconds after the ethernet up then go login screen

ip -6 link set lo up
#ip -6 link set eth0 up
ifconfig eth0 up promisc allmulti

echo "Wait 10 secs..."            
sleep 10                          

10.Finally, fix /usr/local/v6eval/bin/unknown/ping6.rmt

########################################################################
use V6evalRemote;

rOpen() || goto error;

$NUTif = $rOpt_if;
$dstaddress = $rOpt_addr;
$rOpt_timeout = 5 if ! defined $rOpt_timeout;

if($rOpt_size < 1) {
    $rOpt_size = 2;
}

$pingpath = "ping6";
$pingopt  = "-n -c 1 -i 1 -s $rOpt_size -p 00 -w 2 -M want";
$ifopt    = "-I $NUTif" if $NUTif ne "";

rLogin($rOpt_timeout) || goto error;
rCommand("$pingpath $pingopt $ifopt $dstaddress", 15) || goto error;
rLogout($rOpt_timeout) || goto error;

rClose();
exit($V6evalRemote::exitPass);

error:
    rClose();
    exit($V6evalRemote::exitFail);

########################################################################

LeopardBoard DM368 ffmpeg

For the HD video I need libraries to support video / audio file mux. The ffmpeg is a good choice. With ffmpeg I can save H.264 raw data into mp4 file then play video on PC.

1.Download ffmpeg-0.6.1
Download ffmpeg-0.6.1 and put it in ~/LeopardBoard/DM368

2.Cross compile ffmpeg
I will install ffmpeg into the toolchain sub directory for convenience. My toolchain locate at /opt/arm-2009q1

$ cd ~/LeopardBoard/DM368/ffmpeg-0.6.1
$ mkdir build
$ cd build
$ ../configure --target-os=linux --arch=armv --cpu=armv5te --enable-cross-compile --cross-prefix=arm-none-linux-gnueabi- --prefix=/opt/arm-2009q1/arm-none-linux-gnueabi/libc/usr
$ make
$ sudo PATH=$PATH:/opt/arm-2009q1/bin make install

3.Apply patch to fix Makefile to prevent application link error and add new application video_encode_mp4.

ti-dvsdk_dm368-evm_4_02_00_06_dmai_2_20_00_15_ffmpeg.patch

There will be link error while compile application, so I did some modify to dmai_2_20_00_15/packages/ti/sdo/dmai/apps/Makefile.app

The new application is capable to record HD video and save it into mp4 file

4.Recompile DMAI

$ cd ~/LeopardBoard/DM368/original/ti-dvsdk_dm368-evm_4_02_00_06
$ make dmai

The new application video_encode_mp4 is located at dmai_2_20_00_15/packages/ti/sdo/dmai/apps/video_encode_mp4/linux/video_encode_mp4_dm368.x470MV

5.Running
Save 720p video into SD card with file name video_encode.mp4

$ cd /usr/share/ti/ti-dmai-apps
$ ./video_encode_mp4_dm368.x470MV -I camera -y 4 -c h264enc -i /dev/video0 -o /media/mmcblk0p2/video_encode.mp4 -e encode -b 2000000

6.Running with benchmark

$ ./video_encode_mp4_dm368.x470MV -I camera -y 4 -c h264enc -i /dev/video0 -o /media/mmcblk0p2/video_encode.mp4 -e encode -b 8000000 --benchmark
Starting application...
davinci_resizer davinci_resizer.2: RSZ_G_CONFIG:0:1:124
vpfe-capture vpfe-capture: IPIPE Chained
vpfe-capture vpfe-capture: Resizer present
Leopard: switch to HD imager-MT9P031 video input
-----Exposure time = 2f2
-----Exposure time = 2f2
vpfe-capture vpfe-capture: width = 1280, height = 720, bpp = 1
vpfe-capture vpfe-capture: adjusted width = 1280, height = 720, bpp = 1, bytesperline = 1280, sizeimage = 1382400
vpfe-capture vpfe-capture: width = 1280, height = 720, bpp = 1
vpfe-capture vpfe-capture: adjusted width = 1280, height = 720, bpp = 1, bytesperline = 1280, sizeimage = 1382400
Output #0, mp4, to '/media/mmcblk0p2/video_encode.mp4':
    Stream #0.0: Video: 0x0000, yuv420p, 1280x720, q=2-31, 8000 kb/s, 90k tbn, 15 tbc
No frame read ...
...

Frame 26: Read time: 8431us
[27] Encode: 19808us
File write time: 3915us
Total: 32414us

 Frame 27: Read time: 9166us
[28] Encode: 19778us
File write time: 4592us
Total: 33793us

 Frame 28: Read time: 8517us
[29] Encode: 19796us
File write time: 5055us
Total: 33621us

 Frame 29: Read time: 8097us
[30] Encode: 28264us
File write time: 18042us
Total: 54664us

 Frame 30: Read time: 19820us
[31] Encode: 26007us
File write time: 18879us
Total: 64968us

 Frame 31: Read time: 21273us
[32] Encode: 19804us
File write time: 2930us
Total: 44265us

 Frame 32: Read time: 10147us
[33] Encode: 19743us
File write time: 2517us
Total: 32662us
...

Write file time about 2 ~ 5 ms, there's no need to write file in another thread.

LeopardBoard DM368 HD H.264 encode

With DVSDK working on LeopardBoard DM368, Now let's trying video encode function.

1.Check kernel log to see if the dsplink related modules are loaded.

CMEMK module: built on Jun 14 2011 at 18:02:26
  Reference Linux version 2.6.32
  File /home/gigijoe/LeopardBoard/DM368/original/ti-dvsdk_dm368-evm_4_02_00_06/linuxutils_2_26_01_02/packages/ti/sdo/linuxutils/cmem/src/module/cmemk.c
allocated heap buffer 0xc6000000 of size 0x4400000
heap fallback enabled - will try heap if pool buffer is not available
CMEM Range Overlaps Kernel Physical - allowing overlap
CMEM phys_start (0x1000) overlaps kernel (0x80000000 -> 0x83c00000)
cmemk initialized
IRQK module: built on Jun 14 2011 at 18:02:29
  Reference Linux version 2.6.32
  File /home/gigijoe/LeopardBoard/DM368/original/ti-dvsdk_dm368-evm_4_02_00_06/linuxutils_2_26_01_02/packages/ti/sdo/linuxutils/irq/src/module/irqk.c
irqk initialized
EDMAK module: built on Jun 14 2011 at 18:02:30
  Reference Linux version 2.6.32
  File /home/gigijoe/LeopardBoard/DM368/original/ti-dvsdk_dm368-evm_4_02_00_06/linuxutils_2_26_01_02/packages/ti/sdo/linuxutils/edma/src/module/edmak.c

2.If there are errors check /etc/init.d/loadmodule-rc. Below is mine

#!/bin/sh

# Default DM368 EVM Memory Map
#
# Start Addr    Size    Description
# -------------------------------------------
# 0x00001000    32K     ARM TCM memory
# 0x80000000    48 MB   Linux
# 0x83000000    12 MB   Video driver memory (Linux)
# 0x83C00000    68 MB   CMEM
# 0x88000000    BOTTOM  ADDRESS

load () {
    modprobe cmemk phys_start=0x83C00000 phys_end=0x88000000 allowOverlap=1 phys_start_1=0x00001000 phys_end_1=0x00008000 pools_1=1x28672 useHeapIfPoolUnavai
lable=1
    modprobe irqk
    modprobe edmak
    modprobe dm365mmap
    mknod /dev/dm365mmap c `awk "\\$2==\"dm365mmap\" {print \\$1}" /proc/devices` 0
}

unload () {
    rmmod cmemk 2>/dev/null
    rmmod irqk  2>/dev/null
    rmmod edmak 2>/dev/null
    rmmod dm365mmap 2>/dev/null
    rm -f /dev/dm365mmap
}

case "$1" in
      start)
            load
             ;;
       stop)
            unload          
             ;;
       restart)
            unload          
            load
             ;;
        *)
             echo "$0 "
             ;;
esac

3.Decode test to verify if DSP works.

$ cd /usr/share/ti/ti-dmai-apps
$ ./video_decode_io2_dm368.x470MV -c h264dec -e decode -i /usr/share/ti/data/videos/davincieffect.264 -n 30 -o output.yuv
Starting application...
Loader created and primed, entering main loop..
Loaded Frame 1: frame size 401
Loaded Frame 2: frame size 36
Loaded Frame 3: frame size 804
Loaded Frame 4: frame size 844
Loaded Frame 5: frame size 863
Loaded Frame 6: frame size 933
Loaded Frame 7: frame size 1189
Loaded Frame 8: frame size 1253
Loaded Frame 9: frame size 1509
Display Frame 1: Wrote 420P frame size 1382400 (1280x720) to disk
Loaded Frame 10: frame size 1573
Display Frame 2: Wrote 420P frame size 1382400 (1280x720) to disk
Loaded Frame 11: frame size 1829

4.Now, it's time to encode video from camera. I got mt9p031 (LI-BCM5M03).
If you got kernel log below, the camera is ready

...
Leopard: switch to HD imager-MT9P031 video input
mt9p031 1-0048: Detected a MT9P031 chip ID 1801
mt9p031 1-0048: mt9p031 1-0048 decoder driver registered !!
vpfe-capture vpfe-capture: v4l2 sub device mt9p031 registered
...

5.Test if camera works

$ cd /usr/share/ti/ti-dmai-apps
$ ./video_loopback_dm368.x470MV -O component -I camera -y5
davinci_resizer davinci_resizer.2: RSZ_G_CONFIG:0:1:124
vpfe-capture vpfe-capture: IPIPE Chained
vpfe-capture vpfe-capture: Resizer present
Leopard: switch to HD imager-MT9P031 video input
-----Exposure time = 456
-----Exposure time = 456
vpfe-capture vpfe-capture: width = 1920, height = 1080, bpp = 1
vpfe-capture vpfe-capture: adjusted width = 1920, height = 1080, bpp = 1, bytesperline = 1920, sizeimage = 3110400
vpfe-capture vpfe-capture: width = 1920, height = 1080, bpp = 1
vpfe-capture vpfe-capture: adjusted width = 1920, height = 1080, bpp = 1, bytesperline = 1920, sizeimage = 3110400
davinci_v4l2 davinci_v4l2.1: Before finishing with S_FMT:
layer.pix_fmt.bytesperline = 1920,
 layer.pix_fmt.width = 1920,
 layer.pix_fmt.height = 1080,
 layer.pix_fmt.sizeimage =3110400
davinci_v4l2 davinci_v4l2.1: pixfmt->width = 1920,
 layer->layer_info.config.line_length= 1920

6.There's a application can do H.264 RTP which included in my dmai_2_20_00_15 patch called video_encode_rtp.

This application get frames from camera through VPFE then encoded in H.264 format packeted with H264RTP packet then send to network.

From LeopardBoard DM368

$ cd /usr/share/ti/ti-dmai-apps
$ ./video_encode_rtp_dm368.x470MV -I camera -y 4 -c h264enc -i /dev/video0 -o 192.168.168.72 -p 1234 -e encode -b 2000000

-I camera = camera input
-y 4 = 720P,
-o 192.168.168.72 = Send to 192.168.168.72
-p 1234 = port 1234
-b 2000000 = Bit rate 2000000

I'm decoding the video stream with gstreamer on PC Ubuntu.

$ gst-launch udpsrc port=1234 caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H264" ! gstrtpjitterbuffer latency=100 ! rtph264depay ! ffdec_h264 ! xvimagesink

If the video frame sometimes droped, try larger gstrtpjitterbuffer latency = 200 / 300 / 500.

7.Full HD

From LeopardBoard DM368

-y 5 = 1080P

$ ./video_encode_rtp_dm368.x470MV -I camera -y 5 -c h264enc -i /dev/video0 -o 192.168.168.72 -p 1234 -e encode -b 2000000