BPI-R2 EMMC demaged

After installing OS (Stretch from Frank-FW - boot from SD, root FS on 1 partition emmc /f2fs/) on emmc, and a few hours of operation, emmc was destroyed. There was a message about the wo file system, and after reboot emmc is not detected (it is not in / dev) there is only a message in dmsg:

mmc1: error -110 whilst initialising MMC card.

Something can be done?

regards

mt_werty

Can you access it from sdcard or uboot?

What do you mean with /f2fs/,a filesystem? Why not ext4 for rootfs?

SD is working. Message:

mmc1: error -110 whilst initialising MMC card

i get when OS working from SD (boot from SD mmcblk0p1, root FS mmcblk0p2 /f2fs/)

F2FS? Experiment. Subjectively it is faster - I did not make measurements, but on an SD card (also on R.I.P. EMMC it was so) it works noticeably faster. I didn’t care about reliability - I don’t have anything to care about in the event of a breakdown. I didn’t expect a complete EMMC failure - at most problems with FS.

PS and to tell you the truth, I think f2fs is not the cause here - it has been working for several weeks on the SD card as rootfs.

F2FS is a good filesystem to use on flash storage. And I don’t think your eMMC problems are caused/related to F2FS. It’s more probable that you just been hit by Murphy’s law - your eMMC failed because everything fails from time to time. Still it’d be good idea to show us your kernel messages log (dmesg), at least parts related to mmc* - there might be some more messages hiding there you missed out, who knows? Nevertheless if it’s really the case of failed eMMC chip - it is possible to fix by replacing chip. Look for component level repair shops near your location (i.e. guys who repair motherboards, videocards or mobile phones) and ask them for help. It is like a 15 minutes task for experienced person to replace eMMC chip using bottom heater + hot air soldering gun and eMMC chips are dirt cheap if we’re talking about small storage capacities like 8 or 16GBs.

I think so too. but I paste:

# dmesg|grep mmc
> [    0.000000] Kernel command line: board=bpi-r2 console=earlyprintk console=tty1 fbcon=map:0 console=ttyS0,115200 root=/dev/mmcblk0p2 rootfstype=f2fs rootwait vmalloc=496M debug=7 initcall_debug=0 video=1280x1024
> 
> [    5.013733] mtk-msdc 11240000.mmc: GPIO lookup for consumer cd
> 
> [    5.019545] mtk-msdc 11240000.mmc: using device tree for GPIO lookup
> 
> [    5.025934] of_get_named_gpiod_flags: parsed 'cd-gpios' property of node '/mmc@11240000[0]' - status (0)
> [    5.035412] mtk-msdc 11240000.mmc: Got CD GPIO
> 
> [    5.039832] mtk-msdc 11240000.mmc: GPIO lookup for consumer wp
> 
> [    5.045659] mtk-msdc 11240000.mmc: using device tree for GPIO lookup
> 
> [    5.051994] of_get_named_gpiod_flags: can't parse 'wp-gpios' property of node '/mmc@11240000[0]'
> 
> [    5.060724] of_get_named_gpiod_flags: can't parse 'wp-gpio' property of node '/mmc@11240000[0]'
> 
> [    5.069381] mtk-msdc 11240000.mmc: using lookup tables for GPIO lookup
> 
> [    5.075880] mtk-msdc 11240000.mmc: lookup for GPIO wp failed
> 
> [    5.142558] mtk-msdc 11230000.mmc: GPIO lookup for consumer wp
> 
> [    5.148374] mtk-msdc 11230000.mmc: using device tree for GPIO lookup
> 
> [    5.154757] of_get_named_gpiod_flags: can't parse 'wp-gpios' property of node '/mmc@11230000[0]'
> 
> [    5.163508] of_get_named_gpiod_flags: can't parse 'wp-gpio' property of node '/mmc@11230000[0]'
> 
> [    5.172166] mtk-msdc 11230000.mmc: using lookup tables for GPIO lookup
> 
> 
> [    5.178651] mtk-msdc 11230000.mmc: lookup for GPIO wp failed
> 
> [    5.228806] mmc0: host does not support reading read-only switch, assuming write-enable
> 
> [    5.239126] mmc0: new high speed SDHC card at address 0007
> 
> [    5.291156] mmcblk0: mmc0:0007 SD32G 29.0 GiB
> 
> [    5.297160]  mmcblk0: p1 p2
> 
> [    8.357693] F2FS-fs (mmcblk0p2): orphan cleanup on readonly fs
> 
> [    8.383593] F2FS-fs (mmcblk0p2): Mounted with checkpoint version = 1178
> 
> [    8.831677] mmc1: error -110 whilst initialising MMC card
> 
> [   12.071539] mmc1: error -110 whilst initialising MMC card
> 
> [   13.245596] FAT-fs (mmcblk0p1): Volume was not properly unmounted. Some data may be corrupt. 
> Please run fsck.
> 
> [   15.331807] mmc1: error -110 whilst initialising MMC card
> 
> [   18.501538] mmc1: error -110 whilst initialising MMC card

it’s after systems crash

Thanks for sharing. Error -110 is ETIMEDOUT thus I’m afraid it’s probably eMMC chip died on you. Or it might be some other hardware problem like cracked solder joint, e.t.c. Replacing eMMC chip with a new one is your best bet for this case.

Thank you very much for help. :slight_smile:

Looks like I’m having hardware eMMC problems too on one of my BPi R2 boards.

Symptoms are:

  • In u-boot mmc dev 0 either reports nothing (happens in older u-boot version, 2019.01) or behaves like below (version 2020.04):

    BPI-R2> mmc dev 0
    mmc_init: -110, time 99
    BPI-R2> mmc dev 0
    mmc_init: -110, time 97
    
  • In linux /dev/mmcblk0* might work for some time but then the following lines appear in the kernel log and all processes trying to access /dev/mmcblk0* go to deadlock state:

    [   26.439396] mtk-msdc 11230000.mmc: msdc_request_timeout: aborting cmd/data/mrq
    [   26.446570] mtk-msdc 11230000.mmc: msdc_request_timeout: aborting mrq=de5ebcd0 cmd=18
    [   26.454365] mtk-msdc 11230000.mmc: msdc_request_timeout: abort data: cmd18; 384 blocks
    

What is interesting is the fact that I had not used eMMC on this particular board for even a second before the problem manifested. It was a board I bought back in summer 2019 planning to use it as a guinea pig for R2-related development. Upon receiving the parcel with this board I had inspected it thoroughly for damaged or misplaced parts and none were found. Then I did a basic test to confirm that board works: tried to boot my OpenWRT 18.06.2 fork from SD card. It worked normally so I put a mental “checkmark” in the “received board is OK” box and put board aside for quite a while.

Recently I got back into position when I’m able to work from time to time on the R2-related woes thus I started to use this board and stumbled onto these problems with eMMC. Not so sure what to do with it now. Installed eMMC chip is a BGA package which is a nightmare to work with in home hobbyist environment. Probably I would simply count my losses and live on disabling mmc0 in both u-boot and linux DTS files for this board :sob: .

Oh well, it is getting out of hands now. I’ve had yet another spare R2 board laying around which came in the same parcel as a one having eMMC troubles I mentioned in previous post. So I went on and replaced that one with a spare just to confirm that it is not a software problem. What would you thought, eMMC is broken on a spare board as well:

BPI-R2> mmc dev 0
Card did not respond to voltage select!
mmc_init: -95, time 102

As you can see error message is different compared to previous one. Having two boards with failed eMMC in a row forced me to continue testing but the only other R2 board I’ve got in reach is one that I use as a main home server/router meaning that I won’t be having internet access if I take it for tests. It did not stopped me from going on and it was a good decision:

BPI-R2> mmc dev 0
switch to partitions #0, OK
mmc0(part 0) is current device

All conditions are the same but eMMC works without any trouble on this board and fails on other two.

Honestly I don’t know how to comment here. @sinovoip, are there any statistics out there on the eMMC failure rate for R2? When two out of three boards I’ve got on hands right now experience problems with eMMC it looks and smells like a fabric defect for me. Especially keeping in mind that two of these boards are “brand new” with me buying them as a “cold reserve spares” to be laying in storage till their time comes. Either eMMC chips used being low tier/counterfeit/throw-out-binning or there’s something wrong with the chips soldering side of things (bad BGA solder joint, e.t.c.).

Here is photo of the working eMMC chip:

And here are photos of failing chips:

Visible difference is that working chip is an older one while non-working seem to be from a more recent die revision. I had checked datasheets and it seems that old and new dies are expected to be 100% compatible. What makes me a bit suspicious about non-working ones is a fact that markings look different on them: one being bright white and clearly visible while another being totally black and looks like laser-etched on the surface of the chip, so it is only visible when looking in special lighting conditions at certain viewing angles.

@sinovoip @sinovoip1
May I ask you to provide information about the additional contact pads that can be seen to the sides of the eMMC chips? It looks like TSOP48 but AFAIK there were no eMMC chips in TSOP48 packaging. What were these soldering spots designed to be used for?

It looks like I have no other choice except to unsolder existing failing eMMC chips from my boards and replace them with something else. Is it possible to use something in TSOP48 package as a replacement or not? I’m way more comfortable in soldering TSOP chip instead of having to deal with BGA153 packages.

Thanks in advance for your support.

Is first board with samsung emmc a v1.0 board and the other 2 v1.1?

I found a photo of one of your boards here showing you have a v1.2…is this one of the 2 not working?

I wonder why you have issues on linux too…maybe you need the reset-commit there (have added in my 5.4-branch and linked here together with the uboot-patch)