IP04 TESTPLAN David Rowe Created April 2007 This document is a Test Plan/Report for the hardware and software tests used to verify the IP04 from low level hardware functionality, through boot loader, uClinux, and finally Asterisk load testing. The tests are presented in checklist format, and may be useful for others who are bringing up IP04 designs or developing software for the platform. [X] Hardware Only Tests [X] No smoke on power up + well there was some the first time, and a nice explosion! + but OK after caps replaced with 25V version [X] Current drain OK + with 12VDC, 70mA with blank flash [X] 3V3 rail preset [X] 1V2 rail present [X] 10 MHz clock running + probed one side of crysal, also CLKOUT via on LHS of U1 [X] U11 2.048MHz clock [X] Pin 1 U10 3V3 + this indicates BF532 is out of reset [X] 32768 kHz clock running + needed to connect VDDRTC to 3V3 [X] M25P20 SPI flash detected + not detected by jtagprog as explained in README.Rev642 + hacked jtagprog to fix [X] M25P20 SPI flash written and verified OK + this proved PSI flash working OK [ ] u-boot [X] study u-boot code and modify for IP04 [X] first output from RS232 + no prompt, traced to lack of power on RTC [X] reach u-boot prompt [X] read/write env variables + changed bootcmd then powered down/up + shows we can read/write to M25P20 SPI flash [X] Ethernet [X] ping a server + this shows DM9000 address mapping is OK + plus DM9000 hardware loaded OK [X] tftp a u-boot image [X] tftp and run a u-boot image [X] NAND detected [X] NAND read/write OK [X] check waveform on PF10 (NAND READY) net + this is an open drain output and 100k R121 pullup may need to be reduced + looked fine on the scope, 100k is OK, tr=6.7uS tf=800ns, nice clean looking waveform [X] flash u-boot using u-boot + now sure how this works with small SPI flash [ ] integrate ip04 u-boot into astfin [ ] u-boot.ldr generated by astfin works + flash into M25P20 to test [ ] run a memory test + test all SDRAM for a couple of hours [ ] uClinux [X] bootm a uImage + tftp from server, dont store in NAND for now [X] Ethernet works + worked out how to pass ethaddr (MAC) to uClinux from u-boot + forgot to solder R73 (IRQ line) which slowed me down for a while [X] boot uClinux from NAND flash + uImage or kernel + rootfs [X] uClinux detects NAND + cat /proc/mtd works well [X] uClinux reads/writes files to NAND [ ] badblocks test + to determine NAND is reliable [X] save some voicemail to it [X] Run Asterisk from flash + dont use RAM disk [X] root on NAND [X] kernel on NAND [X] * voice prompts in NAND + without buffering in SDRAM + check thay play fast enough [X] integrate into astfin [ ] Asterisk [X] Starts + software only, no FXO/FXS ports [X] Analog ports [X] Detected by wcfxs [X] dialtone on FXS port [X] make call from FXS port + dialled 6000 and hit the menu [X] FXS-FXS call [X] FXS-FXO call + called the time 1194 + sounded nice and clear but low level, need to adjust gains [X] FXS to SIP call [ ] Web GUI [X] comes up + I tried a few things on it, and it looked OK [ ] stable + however it crashed after a while so I think there are some stability probs, perhaps due to running on uClinux [X] config saved to NAND [X] Boot astfin 10 times from NAND [X] prompts + many currently missing, as optional step in astfin based on presence of USB drive (/etc/rc) + could use alaw prompts as we have plenty of space + anyway put them back [X] fully load all 4 analog ports for 24 hrs [X] Call someone via IAX + I called Digium via the 500 exteniosn, sounded nice and clear, fairy low CPU load. [X] clean reboot with wcfxs running + 'reboot' command cuases hang when drivers running + works reliably when wcfxs not loaded TESTPLAN for Astfin/IP04 ------------------------ Please repeat these tests (or a subset thereorf) when you modify the IP04 build of Astfin. The list below was conducted using the 2007R1-RC3 astfin port build (SVN rev 821): [X] uClinux boots [X] zaptel drivers boot + analog ports detected [X] zapscan and ztcfg run [X] Asterisk running (root on ext2) [X] FXS-FXS call [X] Asterisk running (root on yaffs) [X] FXS-FXS call [X] FXS-FXO call [X] incoming FXO calls + call from PSTN [X] FXS to IAX call + used x86 box on LAN for other end of IAX call [X] SIP to FXS call + used wireless SIP phone to make FXS and FXO calls [X] Music on hold + Used wget to place a fpm-calm-river.ulaw to /var/lib/asterisk/moh on IP04 + switched on MOH via Gui (need to click on Activiate Changes) [X] voicemail + saved to NAND flash OK [X] Automated Test #1 - Stability + automated analog and voip calls for 24 hrs + GSM codec for VOIP calls + From May 30 2007 I have been load testing an IP04 with 4 analog and 2 IAX calls for 4 days, about 32,000 calls have been made without any problems + [root@ip04 /etc]$ cat /proc/loadavg 0.16 0.14 0.15 1/43 14315 + no obvious memory leaks + no obvious problems NAND flash (i.e. the file system didn't fill up due to expanding log files) + I have the FXS ports connected to the FXO ports: http://www.rowetel.com/images/ip04/fxs_to_fxo.jpg [X] Automated Test #2 - Max CPU Load + performed 6 June 2007. + designed to put maximum CPU load on system + simulated an installation with 8 SIP phones and an IP04 with 4 analog ports. http://www.rowetel.com/images/ip04/ip04_load_test_2.png + Maximum load was defined as 4 SIP phones connected to 4 analog ports, plus the other 4 SIP phones making IAX or SIP VOIP calls. + For the test an x86 Asterisk box was used to simulate the 8 SIP phones and terminate the SIP/IAX2 VOIP trunks. http://www.rowetel.com/images/ip04/ip04_load_test_3.png + Most of the CPU load (approx 120 MIPs) comes from the echo canceller, which was Oslec configured for 128 tap (16 ms) tail. The tests were arranged so that the echo cancellers were using max CPU load (only far end speech). + This test was run for about an hour with (i) IAX2 VOIP calls and (ii) SIP VOIP calls. GSM was used for the VOIP calls, Alaw between the SIP and analog ports. + So the max load was a total of 8 calls/16 channels. For (i) the loadav was about 2.0 for (ii) the loadave varied between 0.24 and 1.16. This suggests that the system has to work a little harder for IAX2 calls. [X] Network performance + tested using netperf/netserver (blackfin test apps menu) + using 'netperf -H bunny' (bunny was a laptop connected via a wireless lan), throughput was 16.7 Mbit/s, loadav = 0.4 max + using two IP04's connected to the same hub, one running netserver, the other running netperf, obtained 33-41 Mbit/s, loadav = 2.0 max (netserver) and 0.58 (netperf machine) + with two IP04s as per above, 'netperf -H ip04 -t UDP_STREAM -- -m 1024 -l 60' + obtained 50.3 Mbit/s tx, 35.75 Mbit/s rx. + Big question is how to relate these to perforamnce in an IP-PBX role? + I guess at 100 kbit/s per VOIP call perhaps no big deal. + this was really just a characterisation test, no pass or fail conditions [X] reboot reliability + test typing 'reboot' reboots 10/10 times + needed for reliable remote maintenance [ ] GUI can be used to save/recall parameters + reboot to test parameters preserved + GUI failed 4/10 times on ext2, see astfin email "GUI Freeze" + need more work before we can tick this one off + 24/5/07: V0.4 gui much better, occasional freezes in wizard and regular menus. So can't call it a pass yet. I am going to use conf files from now on :-) + I think a long term automated test (for exampl using iMacros) is required to check this one off. + 18/6/07: Tested with asftin trunk rev 893 on IP04, installed on yaffs. 20/20 passes of GUI wizard, much better. This rev uses * 1.4.4. [ ] automated test of GUI + hammer GUI for 24 hrs using GUI test tool like Firefox add-on iMacro + test that it responds and doesn't freeze EVER + this is difficult to verify as the problems have been intermittant [ ] Wear test of Yaffs + characterise yaffs and flash wear problems + take snapshop of /proc/yaffs before + take snapshop of /proc/yaffs after + fill filesystem with copies of a 60 sec vm file + write/read/delete continuosly + try and get some flash wear + relate number of writes to typical useage over time root:/root> cat /proc/yaffs YAFFS built:Jun 16 2007 11:15:31 $Id: yaffs_fs.c 2682 2007-01-22 03:19:29Z aubrey $ $Id: yaffs_guts.c 2682 2007-01-22 03:19:29Z aubrey $ Device 0 "file system" startBlock......... 0 endBlock........... 1983 chunkGroupBits..... 0 chunkGroupSize..... 1 nErasedBlocks...... 1442 nTnodesCreated..... 2200 nFreeTnodes........ 45 nObjectsCreated.... 1500 nFreeObjects....... 60 nFreeChunks........ 107685 nPageWrites........ 0 nPageReads......... 0 nBlockErasures..... 11 nGCCopies.......... 0 garbageCollections. 0 passiveGCs......... 0 nRetriedWrites..... 0 nRetireBlocks...... 0 eccFixed........... 0 eccUnfixed......... 0 tagsEccFixed....... 0 tagsEccUnfixed..... 0 cacheHits.......... 2 nDeletedFiles...... 251 nUnlinkedFiles..... 360 nBackgroudDeletions 0 useNANDECC......... 1 isYaffs2........... 1 root:/root> [ ] Conferencing test + try three extensions test_yaffs_one_file.sh ---------------------- Before: root:/root> cat /proc/yaffs YAFFS built:Jun 16 2007 11:15:31 $Id: yaffs_fs.c 2682 2007-01-22 03:19:29Z aubrey $ $Id: yaffs_guts.c 2682 2007-01-22 03:19:29Z aubrey $ Device 0 "file system" startBlock......... 0 endBlock........... 1983 chunkGroupBits..... 0 chunkGroupSize..... 1 nErasedBlocks...... 1409 nTnodesCreated..... 2600 nFreeTnodes........ 1263 nObjectsCreated.... 1600 nFreeObjects....... 127 nFreeChunks........ 118577 nPageWrites........ 0 nPageReads......... 0 nBlockErasures..... 38 nGCCopies.......... 0 garbageCollections. 38 passiveGCs......... 38 nRetriedWrites..... 0 nRetireBlocks...... 0 eccFixed........... 0 eccUnfixed......... 0 tagsEccFixed....... 0 tagsEccUnfixed..... 0 cacheHits.......... 2 nDeletedFiles...... 324 nUnlinkedFiles..... 397 nBackgroudDeletions 0 useNANDECC......... 1 isYaffs2........... 1 root:/root>