cool tech zone zone
Database is incomplete, how to debug? - Printable Version

+- cool tech zone zone (https://forum.cooltech.zone)
+-- Forum: Tangara (https://forum.cooltech.zone/forumdisplay.php?fid=3)
+--- Forum: General (https://forum.cooltech.zone/forumdisplay.php?fid=5)
+--- Thread: Database is incomplete, how to debug? (/showthread.php?tid=79)



Database is incomplete, how to debug? - js - 2024-12-19

Hi,

I received my Tangara yesterday, and have been playing around with it. First I flashed the 1.0.1 firmware, then I copied my moderately large library over. Tangara created its database automatically.

However, I noticed the database is missing several tracks/albums. I can listen to those through the file browser feature, but they don't show up in the indexed views.

My best guess is that one of my tracks has a tag that Tangara doesn't like, and it aborts the database scan early? I have no direct proof, but the albums I noticed are missing are ones with later file creation dates (newer additions to the library) so I suspect they would be scanned last. It's possible there are older albums also missing, but I haven't noticed any yet. I manage all my music through beets so I think all my tags should be more or less consistent.

I'm trying to debug this issue, but I'm not sure how to get the data I need about which tracks are missing. What is the best way to inspect the Tangara database contents? I might be able to do a diff with my beets track list and figure out where it's choking, if my theory is correct. My LOG and LOG.old files are both empty - 0 bytes. I tried getting the info through the Lua console in Tangara Desktop Companion but anything more complex than a one-liner gives me Command returned non-zero error code: 0x1 (ERROR). Even running t = true in the console gives this reply. My database schema version, according to Tangara, is 8.


Would love some help debugging this!

Thanks!

- JS

PS: very minor, but my Tangara Companion is showing the database file size incorrectly as X GiB instead of X KiB.


RE: Database is incomplete, how to debug? - jacqueline - 2024-12-20

Hi!

So first question: is your music just in the root of your SD card, or is it in a Music/ directory? The next firmware release (coming soon later today!) will help with tracks not in a Music/ directory not being picked up.

Otherwise, if you keep Tangara plugged in + the serial port it exposes open during indexing, it should log any tag parsing errors it encounters whilst indexing your tracks. These could be useful for tracking down why some of your tracks are showing up but others aren't.

On your PS, this should be fixed at head I think: https://github.com/haileys/tangara-companion/commit/f87b2350eaf81be9ed263a95e800275ddfe2b29b


RE: Database is incomplete, how to debug? - js - 2024-12-20

(2024-12-20, 12:09 AM)jacqueline Wrote: Hi!

So first question: is your music just in the root of your SD card, or is it in a Music/ directory? The next firmware release (coming soon later today!) will help with tracks not in a Music/ directory not being picked up.

Otherwise, if you keep Tangara plugged in + the serial port it exposes open during indexing, it should log any tag parsing errors it encounters whilst indexing your tracks. These could be useful for tracking down why some of your tracks are showing up but others aren't.

On your PS, this should be fixed at head I think: https://github.com/haileys/tangara-companion/commit/f87b2350eaf81be9ed263a95e800275ddfe2b29b

Hi jacqueline! Thanks for getting back to me so fast.

My music is one folder per album inside a "/music" (lowercase) directory.

I have just started a database update with the serial port open. So far the only logs are related to missing TTS items (a feature I was thinking I might have to develop, but it's 1000% awesome it already exists!!) however the scan has just started so I will leave it running and report back if I find anything useful.

I also very briefly played around with the Lua repl over usb serial and it seemed to work better than my experience with the Lua console in the desktop app. So that's now a viable fallback option for investigation if I don't see anything interesting come up in the logs.

On top of being an awesome consumer device to begin with, it is legitimately SO COOL that I'm able to do this with Tangara. It's really something special, and I'm so lucky to have one. Thanks for creating it!


RE: Database is incomplete, how to debug? - js - 2024-12-20

I've done some more troubleshooting, but unfortunately still haven't found a solution.

The first thing I did was double check the tags on some of the files I noticed aren't showing up in my database. They look fine to me. They also show up on my phone just fine, so I don't think it's a tagging issue.

Then I ran the database update with the serial console open like you suggested. I only saw 2 things:

1. A bunch of warnings trying to parse cover.jpg in every folder -- which I guess is to be expected since those aren't music tracks.
2. Some warnings of a hash collision. This is also not unexpected, since I have 3 albums in different folders that are, to MusicBrainz, the "same album."

I removed the offending hash collision files and re-scanned but no new logs appeared beyond the cover.jpg ones, and the tracks are still missing.

I tried running the following in the serial Lua repl, to get a dump of all album titles in the database. I'd like to at least narrow down how many / which albums are missing other than the ones I happened to manually check:

Code:
do
    local albums = database.indexes()[4]

    for album in albums:iter() do
        print(album)
    end
end

But it crashed Tangara (out of memory?)

Code:
Guru Meditation Error: Core  1 panic'ed (Unhandled debug exception).
Debug exception reason: Stack canary watchpoint triggered (console_repl)
Core  1 register dump:
PC      : 0x40230762  PS      : 0x00060136  A0      : 0x80231139  A1      : 0x3ffe0510
A2      : 0x00000002  A3      : 0x3ffe0634  A4      : 0x00000000  A5      : 0x00000000
A6      : 0x00000010  A7      : 0x00000001  A8      : 0xc8e9260b  A9      : 0x3ffc3ca4
A10     : 0x3f4419a8  A11     : 0xffffffff  A12     : 0x00000001  A13     : 0x00000000
A14     : 0x00001034  A15     : 0x3ffb1b40  SAR     : 0x00000008  EXCCAUSE: 0x00000001
EXCVADDR: 0x00000000  LBEG    : 0x4000c28c  LEND    : 0x4000c296  LCOUNT  : 0x00000000


Backtrace: 0x4023075f:0x3ffe0510 0x40231136:0x3ffe0630 0x4022eb6d:0x3ffe0670 0x401ccf71:0x3ffe06d0 0x401c66c9:0x3ffe06f0 0x401c9c79:0x3ffe0710 0x401ca5bd:0x3ffe0770 0x400e9bda:0x3ffe0830 0x40224500:0x3ffe18d0 0x4021f868:0x3ffe1930




ELF file SHA256: 8a2a205cf

Rebooting...
ets Jul 29 2019 12:21:46

rst:0xc (SW_CPU_RESET),boot:0x1f (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:5128
load:0x40078000,len:15780
load:0x40080400,len:4
load:0x40080404,len:3700
entry 0x400805c4
W (39) esp_psram: Virtual address not enough for PSRAM, map as much as we can. 4MB is mapped
W (51) i2c: This driver is an old driver, please migrate your application code to adapt `driver/i2c_master.h`
W (1062) i2s_common: dma frame num is out of dma buffer size, limited to 1023
W (1062) i2s_common: dma frame num is out of dma buffer size, limited to 1023

Then I decided to try to take a look at the .tangara-db directory contents. Mine looks like this:

Code:
.rwxr-xr-x  153 js 31 Dec  1979 000005.ldb
.rwxr-xr-x 2.1M js 31 Dec  1979 000595.ldb
.rwxr-xr-x 2.1M js 31 Dec  1979 000596.ldb
.rwxr-xr-x 1.3M js 31 Dec  1979 000597.ldb
.rwxr-xr-x  130 js  1 Jan  1980 000617.ldb
.rwxr-xr-x  124 js 31 Dec  1979 000620.ldb
.rwxr-xr-x    0 js 31 Dec  1979 000631.log
.rwxr-xr-x   16 js 31 Dec  1979 CURRENT
.rwxr-xr-x    0 js 31 Dec  1979 LOG
.rwxr-xr-x    0 js 29 Nov  1979 LOG.old
.rwxr-xr-x  606 js 31 Dec  1979 MANIFEST-000630

I ran cat *.ldb | strings and searched for the missing track/album data, and they're in there! It has the full file paths of the tracks, the album titles, and the song titles.

So, I'm still not sure why they're not appearing in the All Albums, All Tracks, or Albums by Artist lists.

I'm open to other suggestions about what to try next. Later tonight, I am planning to wipe my SD card and start fresh, with a smaller subset of my library including these albums to see if they show up then. If that still fails, I'll try converting all the file formats to mp3 or ogg and see if that changes anything.


RE: Database is incomplete, how to debug? - ailurux - 2024-12-20

Hi, thanks for the detailed info, this really helps. We've had a few similar reports of database indexing not working for tracks it should work with, but this is the first I've seen of the database having the entries and it not showing up, that's very odd. Would you mind sharing the entire database as a zip, or however is easiest for you? It might shed some light on these issues to see what entries are there and if there's any reason they shouldn't show up on the device.

The repl error implies a stack overflow, which isn't uncommon for the repl as it runs with less stack available than the lua in the firmware. I tried it myself and got the same issue, though the albums do show correctly for me on the device, so it's unlikely to be the cause itself.


RE: Database is incomplete, how to debug? - js - 2024-12-20

I tried this again with a minimal library containing one known working album and one known not working album. I backed up then deleted the .tangara-db folder for my full library, cleared out my /music directory except for two albums, and generated a new database with the new smaller library. The same issue is still happening.

The two albums I kept are Angel Olsen's "My Woman" (working) and Magdalena Bay's "Imaginal Disk" (not working).

I was able to run the Lua script successfully this time, and the only album name it printed was "My Woman". But once again, inspecting the database strings shows Imaginal Disk tracks, paths, and the Magdalena Bay artist.

I'm attaching screenshots from Tangara of the various views as well as the new minimal database as a zip file. I am still on the 1.0.1 firmware.

Here is the database zip:
.zip   db.zip (Size: 5.34 KB / Downloads: 1)

The files view, with both folders present:
   

The All Albums view, with only My Woman present:
   

The Albums by Artist view, with only Angel Olsen present:
   


RE: Database is incomplete, how to debug? - ailurux - 2024-12-21

Ok I had a look through the database you sent me (thanks!), and it seems that it might be an issue with the tagging for those tracks. There doesn't seem to be album or artist tags that we're able to read for those ones, so they don't show up in the album or artist views. I can see the database does know about those tracks though, can you check if they show up in "All Tracks"?

I noticed that the filepaths for these tracks do contain the album and artist, and in future we want to add better support for tagging tracks based on common filepath structures (ie, we can make an educated guess for artist and album if the filepath looks like "Artist/Album/01 Track.flac" or "Artist - Album/01 Track.flac"). But at the moment for a track to show up in the album or artist indexes, we rely on tags in the file itself.


RE: Database is incomplete, how to debug? - js - 2024-12-21

They don't appear in All Tracks either.

I've checked the tags and they really look okay to me, but maybe there's something subtle I'm missing.

Here's the ffprobe for Magdalena Bay's "She Looked Like Me!" - it appears to have proper Artist and Title tags as far as I can tell.

Code:
ffprobe version n7.0.1 Copyright (c) 2007-2024 the FFmpeg developers
  built with gcc 14.1.1 (GCC) 20240522
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-frei0r --enable-gmp --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libdav1d --enable-libdrm --enable-libdvdnav --enable-libdvdread --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libharfbuzz --enable-libiec61883 --enable-libjack --enable-libjxl --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libplacebo --enable-libpulse --enable-librav1e --enable-librsvg --enable-librubberband --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpl --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-mbedtls --enable-nvdec --enable-nvenc --enable-opencl --enable-opengl --enable-shared --enable-vapoursynth --enable-version3 --enable-vulkan
  libavutil      59.  8.100 / 59.  8.100
  libavcodec     61.  3.100 / 61.  3.100
  libavformat    61.  1.100 / 61.  1.100
  libavdevice    61.  1.100 / 61.  1.100
  libavfilter    10.  1.100 / 10.  1.100
  libswscale      8.  1.100 /  8.  1.100
  libswresample   5.  1.100 /  5.  1.100
  libpostproc    58.  1.100 / 58.  1.100
Input #0, flac, from 'Magdalena Bay - Imaginal Disk/01 She Looked Like Me!.flac':
  Metadata:
    ALBUM           : Imaginal Disk
    ALBUM ARTIST    : Magdalena Bay
    album_artist    : Magdalena Bay
    ALBUMARTIST_CREDIT: Magdalena Bay
    ALBUMARTISTSORT : Magdalena Bay
    ALBUMARTISTS    : Magdalena Bay
    ALBUM_ARTISTS   : Magdalena Bay
    ALBUMARTISTS_CREDIT: Magdalena Bay
    ALBUMARTISTS_SORT: Magdalena Bay
    RELEASESTATUS   : Official
    MUSICBRAINZ_ALBUMSTATUS: Official
    RELEASETYPE     : album
    MUSICBRAINZ_ALBUMTYPE: album
    ARTIST          : Magdalena Bay
    ARTIST_CREDIT   : Magdalena Bay
    ARTISTSORT      : Magdalena Bay
    ARTISTS         : Magdalena Bay
    ARTISTS_CREDIT  : Magdalena Bay
    ARTISTS_SORT    : Magdalena Bay
    BARCODE         : 810090095448
    BPM             : 0
    COMPILATION     : 0
    RELEASECOUNTRY  : XW
    DATE            : 2024-08-23
    YEAR            : 2024
    disc            : 1
    DISCTOTAL       : 1
    DISCC           : 1
    TOTALDISCS      : 1
    ISRC            : USQE92400094
    LABEL           : Mom+Pop
    PUBLISHER       : Mom+Pop
    LANGUAGE        : eng
    LYRICS          : [Intro]
                    : Hello
                    :
                    : [Verse 1]
                    : Down the line, over the waves
                    : Two kids and a military
                    : Turn their tongue, change their name
                    : La love, born to marry
                    : Crossed their hearts, crossing the earth
                    : One year, then a baby's carried
                    : Grows up young, screams at graves
                    : Bang-bang and it's customary
                    : Ordinary
                    :
                    : [Verse 2]
                    : Hang the war over the hearth
                    : Five more in a house of many
                    : Argentine Fabergé
                    : La love, born unsteady
                    : Kid hears ghosts singing his name
                    : Likes guns and he's pistol-ready
                    : That's his curse, that's my name
                    : Hе sings up to the moon unchaining
                    :
                    : [Verse 3]
                    : I didn't know her when shе walked in
                    : But I recognized her eyes
                    : Her hair lifted as she stood still
                    : Right there in the light
                    : And she felt like a dream
                    : I didn't know what to think
                    : She looked like me
                    : We stood mirrored 'til I leaned in
                    : And then you know what she did?
                    : She shot at me like an earthbound bullet
                    : And then she wrapped her hands around my neck and I felt love
                    : Just ordinary love
                    : Let me hold you in my arms
                    : Oh, love, that old familiar drug
                    :
                    : [Outro]
                    : Bang-bang and it's customary
                    : America stole my fate
                    : La love and a baby carriage
                    : Turned my tongue, changed my name
                    : Bang-bang, born unsteady
                    : Crossed my heart, crossed the earth
                    : La love and a stone to carry
                    : That's my curse, that's my name
                    : Bang-bang and it's customary
                    : Ordinary
    MUSICBRAINZ_ALBUMARTISTID: 68c261d5-48c8-4ea9-9466-9fc908dc79bb
    MUSICBRAINZ_ALBUMID: acad6b60-2b75-425a-ae50-0900ac45dda4
    MUSICBRAINZ_ARTISTID: 68c261d5-48c8-4ea9-9466-9fc908dc79bb
    MUSICBRAINZ_RELEASEGROUPID: 9c710cb4-6b20-47d7-a31a-5adb064c6c6b
    MUSICBRAINZ_RELEASETRACKID: 58541661-1306-4ef3-b487-e291eb08bb0f
    MUSICBRAINZ_TRACKID: 70487029-1d32-4b7a-ba2a-acfe2462f312
    MUSICBRAINZ_WORKID: 461b9b0f-6b17-469e-8d78-d5801cdd1731
    MEDIA           : Digital Media
    ORIGINALDATE    : 2024-08-23
    REPLAYGAIN_ALBUM_GAIN: -9.30 dB
    REPLAYGAIN_ALBUM_PEAK: 0.995656
    REPLAYGAIN_TRACK_GAIN: -9.09 dB
    REPLAYGAIN_TRACK_PEAK: 0.992118
    SCRIPT          : Latn
    TITLE           : She Looked Like Me!
    track           : 1
    TRACKTOTAL      : 15
    TRACKC          : 15
    TOTALTRACKS     : 15
  Duration: 00:03:13.19, start: 0.000000, bitrate: 1632 kb/s
  Stream #0:0: Audio: flac, 44100 Hz, stereo, s32 (24 bit)
      Side data:
        replaygain: track gain - -9.090000, track peak - 0.000023, album gain - -9.300000, album peak - 0.000023,
  Stream #0:1: Video: mjpeg (Baseline), yuvj444p(pc, bt470bg/unknown/unknown), 500x500 [SAR 1:1 DAR 1:1], 90k tbr, 90k tbn (attached pic)
      Metadata:
        comment         : Cover (front)

Here's the ffprobe for Angel Olsen's "Intern":
Code:
ffprobe version n7.0.1 Copyright (c) 2007-2024 the FFmpeg developers
  built with gcc 14.1.1 (GCC) 20240522
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-frei0r --enable-gmp --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libdav1d --enable-libdrm --enable-libdvdnav --enable-libdvdread --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libharfbuzz --enable-libiec61883 --enable-libjack --enable-libjxl --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libplacebo --enable-libpulse --enable-librav1e --enable-librsvg --enable-librubberband --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpl --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-mbedtls --enable-nvdec --enable-nvenc --enable-opencl --enable-opengl --enable-shared --enable-vapoursynth --enable-version3 --enable-vulkan
  libavutil      59.  8.100 / 59.  8.100
  libavcodec     61.  3.100 / 61.  3.100
  libavformat    61.  1.100 / 61.  1.100
  libavdevice    61.  1.100 / 61.  1.100
  libavfilter    10.  1.100 / 10.  1.100
  libswscale      8.  1.100 /  8.  1.100
  libswresample   5.  1.100 /  5.  1.100
  libpostproc    58.  1.100 / 58.  1.100
Input #0, flac, from 'Angel Olsen - My Woman/01 Intern.flac':
  Metadata:
    ACOUSTID_FINGERPRINT: AQADtFESZVGiLJkgwviI48eP_8gP58ihH7_h_EOcHwdz7vhu5NBvPCzeISd6wzFyHMKP__DRXbg36GCvIT_KH58Pf_hx6fgXohJ_5PgP9TraRYd__DCMQycDHj_SBP6O9MUNrTF-hAeTbjl-4_g13HjyDPnQQ3wFH9WhD9yJBj_uHC7-GL-gX3gpxCo64brRW_AlhMSNvoZYZagO7cIPH-IDGMfhHqJyEOmNHtPRJ8cj5EGZQMuhPzmeIy8K9_D
    ACOUSTID_ID     : 94958474-678a-4ac8-b542-3ae29bc8b58b
    ALBUM           : My Woman
    ALBUM ARTIST    : Angel Olsen
    album_artist    : Angel Olsen
    ALBUMARTIST_CREDIT: Angel Olsen
    ALBUMARTISTSORT : Olsen, Angel
    ALBUMARTISTS    : Angel Olsen
    ALBUM_ARTISTS   : Angel Olsen
    ALBUMARTISTS_CREDIT: Angel Olsen
    ALBUMARTISTS_SORT: Olsen, Angel
    RELEASESTATUS   : Official
    MUSICBRAINZ_ALBUMSTATUS: Official
    RELEASETYPE     : album
    MUSICBRAINZ_ALBUMTYPE: album
    ARTIST          : Angel Olsen
    ARTIST_CREDIT   : Angel Olsen
    ARTISTSORT      : Olsen, Angel
    ARTISTS         : Angel Olsen
    ARTISTS_CREDIT  : Angel Olsen
    ARTISTS_SORT    : Olsen, Angel
    BPM             : 0
    COMPILATION     : 0
    COMPOSER        : Angel Olsen
    COMPOSERSORT    : Olsen, Angel
    RELEASECOUNTRY  : US
    DATE            : 2016-09-02
    YEAR            : 2016
    disc            : 1
    DISCTOTAL       : 1
    DISCC           : 1
    TOTALDISCS      : 1
    GENRE           : Rock
    ISRC            : US38Y1628401
    LABEL           : Jagjaguwar
    PUBLISHER       : Jagjaguwar
    LANGUAGE        : eng
    LYRICIST        : Angel Olsen
    MUSICBRAINZ_ALBUMARTISTID: 185527bf-c293-4c24-8213-ed98fb8976be
    MUSICBRAINZ_ALBUMID: a40b984e-ce94-4b1b-9de5-540a75ae1d9b
    MUSICBRAINZ_ARTISTID: 185527bf-c293-4c24-8213-ed98fb8976be
    MUSICBRAINZ_RELEASEGROUPID: c9db1003-a83a-49b7-b04c-237e05260969
    MUSICBRAINZ_RELEASETRACKID: 5cf533d4-11d8-45cd-98b6-450ea40f81ec
    MUSICBRAINZ_TRACKID: eea9c9a2-3b88-4930-a072-55d8a3dcdff0
    MUSICBRAINZ_WORKID: 668e27eb-0a8c-4b87-92da-4c59fa53070c
    MEDIA           : Digital Media
    ORIGINALDATE    : 2016-09-02
    REPLAYGAIN_ALBUM_GAIN: -6.76 dB
    REPLAYGAIN_ALBUM_PEAK: 0.989410
    REPLAYGAIN_TRACK_GAIN: -6.15 dB
    REPLAYGAIN_TRACK_PEAK: 1.000000
    SCRIPT          : Latn
    TITLE           : Intern
    track           : 1
    TRACKTOTAL      : 10
    TRACKC          : 10
    TOTALTRACKS     : 10
  Duration: 00:02:46.72, start: 0.000000, bitrate: 921 kb/s
  Stream #0:0: Audio: flac, 44100 Hz, stereo, s16
      Side data:
        replaygain: track gain - -6.150000, track peak - 0.000023, album gain - -6.760000, album peak - 0.000023,
  Stream #0:1: Video: mjpeg (Baseline), yuvj444p(pc, bt470bg/unknown/unknown), 500x500 [SAR 300:300 DAR 1:1], 90k tbr, 90k tbn (attached pic)
      Metadata:
        comment         : Cover (front)

I diffed the two outputs, and obviously the contents of the tags are not shared between the two - but the only potentially notable differences I noticed are that Angel Olsen has a "COMPOSER" and "COMPOSERSORT" field which Magdalena Bay doesn't. Same for "GENRE."

Magdalena Bay has "LYRICS" that Angel Olsen doesn't. I tried removing that tag and it made no difference to Tangara.


RE: Database is incomplete, how to debug? - chrisw_b - 2024-12-21

Oh I think I've come across a similar issue. Funnily enough I had no problems with that Magdelena Bay album, nor anything else I had saved locally as a flac from Bandcamp. However, an album that I (sadly) bought off amazon, which came as MP3 files, for some reason did not show up when imported to the tangara with the `beet convert -d /PATH/TO/TANGARA/music` command, and the recommended
```
Code:
convert:
  embed: no
  format: opus
settings.

The files did show up fine when I added `never_convert_lossy_files: true`, and they were simply copied as mp3s instead of converted to opus files, which is a good enough solution for me, so no real complaint here

Here's the output of one of the original MP3 files from Gapless

Code:
/run/user/1000/doc/ea77cd6b/01 Over You.mp3
Title=Over You
Artist=HOLYCHILD
Album=The Theatrical Death of Julie Delicious
Album Artist=HOLYCHILD
Composer=Louis Charles Diller
Genre=Electronic, Pop
Datetime=2019-11-08
Track Number=1/10
Disc Number=1/1
Comment=Amazon.com Song ID: 200000461103403
Extended Comment=ID3v1 Comment[en]=Amazon.com Song ID: 20000046
Artist Sortname=HOLYCHILD
Audio Codec=MPEG-1 Layer 3 (MP3)
Bitrate=257000
Channel Mode=stereo
Container Format=ID3 tag
Has Crc=false
Maximum Bitrate=223868
Minimum Bitrate=223868
Album Artist ID=738800f7-d90b-4ee7-9100-b870319d880f
Album ID=b9675ff8-2556-4f91-8bb6-2b093bbec2a4
Artist ID=738800f7-d90b-4ee7-9100-b870319d880f
Beats Per Minute=0
Copyright=(C) 2019 Glassnote Entertainment Group LLC
ISRC=USYAH1900084
Publisher=Glassnote
Release Group ID=e72028ca-8a01-476e-af4c-097ef4536df0
Release Track ID=0b2e5d0d-c66f-4e6f-ad0f-ccf7e765964f
Track ID=ac25a369-a188-4c34-8f77-12a34444a6c9

Hopefully there's some overlap there and it helps track down what's going on with the database, but let me know if there's more info that might be helpful!


RE: Database is incomplete, how to debug? - js - 2024-12-21

(2024-12-21, 05:43 PM)chrisw_b Wrote: Funnily enough I had no problems with that Magdelena Bay album, nor anything else I had saved locally as a flac from Bandcamp.

Huh, that's weird. Because both the albums I chose were downloaded as flac from Bandcamp.

The working Angel Olsen album was downloaded several years ago vs. the not-working Magdalena Bay album which was downloaded this year. Don't know if that helps or just causes more confusion!


RE: Database is incomplete, how to debug? - ailurux - 2024-12-21

This is really helpful, thanks! I vaguely recall we were having some issues with tracks with LYRICS tags a while back but I believe that's since been fixed, and you removed the tag anyway so it's probably not that.

Oh! I think I might have a idea. I noticed the track that failed to index has no genre tag. We fixed one bug relating to this already, where if you have no genre tag and you have no specific media type directories (ie, "Music/", "Audiobooks/", "Podcasts/"), then we just assume it's music. *But* I've just reviewed it and this introduced a new bug where if you *do* have a Music/ directory and there's no genre tags, it doesn't assume it's music. I'll fix that this week. Oops.

If that is the cause in this case, you should be able to fix it by either:
- putting all tracks in the top level and deleting the "Music/" folder (as well as any other media type folders)
- adding a genre tag (literally anything would work so long as it's not empty)

Also noticed another issue that "Music/" is indeed case sensitive here. So it's actually more likely related to that, because it checks that before checking the genre tag.

I'll also fix that this week, it shouldn't be case sensitive.


RE: Database is incomplete, how to debug? - js - 2024-12-21

(2024-12-21, 09:37 PM)ailurux Wrote: Also noticed another issue that "Music/" is indeed case sensitive here. So it's actually more likely related to that, because it checks that before checking the genre tag.

This worked for me with the "small 2 album library"! Renaming the directory from "music" to "Music" has the tracks showing up as expected now.

I will try the same with the full library, but it will take much longer to scan, so I won't be able to provide a quick update. But this does seem like the easiest fix at the moment.


RE: Database is incomplete, how to debug? - chrisw_b - 2024-12-22

(2024-12-21, 09:37 PM)ailurux Wrote: Also noticed another issue that "Music/" is indeed case sensitive here. So it's actually more likely related to that, because it checks that before checking the genre tag.

whoops! I'm too used to making directories lowercase on case sensitive systems (...and also couldn't remember/didn't check if exFAT was case sensitive)


RE: Database is incomplete, how to debug? - jacqueline - 2024-12-29

exFAT is case insensitive, but case preserving. It's annoying. We've fixed this at head now, so v1.1 at the end of this week should no longer have issues with media directories that have unexpected cases.