[READ-ONLY] Mirror of https://github.com/FoxxMD/multi-scrobbler. Scrobble plays from multiple sources to multiple clients
docs.multi-scrobbler.app
deezer
docker
jellyfin
koito
lastfm
listenbrainz
maloja
mopidy
mpris
music
music-assistant
plex
scrobble
self-hosted
spotify
subsonic
tautulli
youtube-music
docs: Add duplicate scrobble detection explanation
+65
+65
docsite/docs/configuration/clients/clients.mdx
+65
docsite/docs/configuration/clients/clients.mdx
···
23
23
24
24
## Features
25
25
26
+
### Duplicate Detection
27
+
28
+
Before a Play is scrobbled to a client MS checks existing scrobbles from the Client's API to see if the Play has already been scrobbled.
29
+
30
+
For each Play, MS fetches (cached) scrobbles from the Client in a time range inclusive of the Play's listening timestamp and then scores existing scrobbles against the Play based on:
31
+
32
+
* Similarity of Title, Artists, and Album
33
+
* Temporal closeness of the Play's timestamp to the existing scrobble's timestamp
34
+
* Whether MS detected the Play as a repeat (applicable to Sources that report realtime Player data)
35
+
36
+
Detailed scoring breakdowns against each existing scrobble are logged at the `TRACE` level.
37
+
38
+
<details>
39
+
40
+
<summary>Detailed Explanation</summary>
41
+
42
+
A **candidate** (to be scrobbled) Play is first transformed using the configured [`compare.candidate` Hook](/configuration/transforms/#hook), if any exists.
43
+
44
+
Next, MS checks (up to) the last 100 scrobbles *in-memory* scrobbles that it has made. These are *not* from the Client but the actual scrobbles MS made while it has been running. The data from these scrobbles is much richer than what is usually parsed from the Client which makes it easier to detect duplicates from.
45
+
46
+
If no in-memory scrobble matches then MS starts comparing the candidate against historical scrobbles fetched from an inclusive time range of the candidate's timestamp.
47
+
48
+
<h3>Matching Title/Artist/Album</h3>
49
+
50
+
For these string-based values MS uses an [token-order-invariant](https://github.com/FoxxMD/multi-scrobbler/blob/0dfa4c7aad6df98e13aee6d395827665c2414adb/src/backend/utils/StringUtils.ts#L299) method that scores string similarity based on a (token count) weighted average of two [similarity](https://foxxmd.github.io/string-sameness/#md:strategies) algorithms, [Levenshtien Distance](https://en.wikipedia.org/wiki/Levenshtein_distance) and [Dice's Coefficient](https://en.wikipedia.org/wiki/S%C3%B8rensen%E2%80%93Dice_coefficient). The weighted average ensures that very long strings (Track titles) are scored with a confidence proportional to their length.
51
+
52
+
<h3>Matching Timestamp</h3>
53
+
54
+
Timestamps are scored on how temporally close they are. There are four possible scores with decreasing value:
55
+
56
+
* **Exact** - Timestamps are within 1 second of each other
57
+
* **Close** - Timestamps are within `threshold` seconds of each other
58
+
* This is determined by the smallest update interval of Source
59
+
* Some Sources (subsonic) only update every 60 seconds so this is the smallest "close" value possible. Most are 10 seconds. Signified by `(Needed <10s)` in the breakdown example below.
60
+
* **Fuzzy** - One timestamp is within `threshold` seconds of the *end* of the other timestamp
61
+
* Sources can set the scrobble timestamp at different times. Some do it when the track is started listening to, some when it ends, some when the *player* stops.
62
+
* Where possible, MS knows and keeps track of when this timestamp *should* be, for each Source. If it's not possible then Fuzzy may be allowed.
63
+
* **None** - There is no correlation between timestamps
64
+
65
+
<h3>Scoring and Breakdows</h3>
66
+
67
+
A candidate Play must score >= 1 to be detected as a duplicate of an existing scrobble.
68
+
69
+
Each score and a breakdown of the scores for its individual components can be see at the `TRACE` logging level or in the [debug data](/help#copy-play-debug-data) for a scrobble. An example:
70
+
71
+
```
72
+
* Artist: 0.06 * 0.3 = 0.02
73
+
* Title: 0.04 * 0.4 = 0.02
74
+
* Time: (Exact) 1 * 0.5 = 0.50
75
+
* Existing: 19:13:33-04:00 - Candidate: 19:13:33-04:00
76
+
* Temporal Sameness: Exact
77
+
* Play Diff: 0s (Needed <10s)
78
+
* Range Comparison N/A
79
+
Score 0.54 => No Match
80
+
```
81
+
82
+
In each component equation, the first number is the similarity (or temporal closeness).
83
+
84
+
* 0 = no correlation
85
+
* 1 = exactly the same
86
+
87
+
The second number is the *weight* of that component in the final score.
88
+
89
+
</details>
90
+
26
91
### Dead Scrobbles
27
92
28
93
If multi-scrobbler is unable to submit a scrobble to a Client then it places the scrobble into a queue which is retried every 5 minutes for a number of times before it gives up.