DFSG NEW Queue

DFSG, Licensing & New Packages Team

Review: golang-github-saintfish-chardet 0.0~git20230101.5e3ef4b+dfsg-1

Package Information

Descriptioncharacter set detection library ported from ICU (library)

Character set detection library for Go, ported from the detection algorithm and statistical data of the International Components for Unicode (ICU) project. Given a byte slice it returns ranked guesses of the text's encoding and language, each with a confidence score.

Separate detectors are provided for plain text and for HTML, the latter ignoring markup when scoring. Recognisers cover the single-byte ISO-8859 and windows-125x families, UTF-8, UTF-16/32 with and without BOM, ISO-2022, and the CJK multi-byte encodings Big5, GB18030, EUC-JP, EUC-KR and Shift_JIS.

MaintainerDebian Go Packaging Team <team+pkg-go@tracker.debian.org>
Changed ByJuan Manuel Méndez Rey <juan.mendezr@proton.me>
Sponsortsimonq2@debian.org
Distributionunstable
Architectureall
VCSgit: https://salsa.debian.org/go-team/packages/golang-github-saintfish-chardet.git (browse)
Closes#1148635
Trackerhttps://tracker.debian.org/pkg/golang-github-saintfish-chardet
Uploaded4 hours ago

New Package Report

.changes◂
Changed-ByJuan Manuel Méndez Rey <juan.mendezr@proton.me>
Architecturesource all
Distributionunstable
DateTue, 22 Sep 2026 02:39:05 +0200
Sourcegolang-github-saintfish-chardet
Version0.0~git20230101.5e3ef4b+dfsg-1
Changelog◂
golang-github-saintfish-chardet (0.0~git20230101.5e3ef4b+dfsg-1) unstable; urgency=medium
 .
   * Initial release (Closes: #1148635)
.dsc◂
Package-Listgolang-github-saintfish-chardet-dev deb golang optional arch=all
Sectiongolang
Priorityoptional
Componentmain
debian/copyright◂
Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Source: https://github.com/saintfish/chardet
Upstream-Name: chardet
Upstream-Contact: yusheng.sjtu@gmail.com
Files-Excluded:
 testdata
Comment: The excluded directory holds 13 scraped copies of live third-party web
 pages (Amazon, MercadoLibre, Orange.fr, Virk.dk, Wikipedia and others), about
 1.1 MB, carried as encoding-detection fixtures with no licence grant of any
 kind. The same content is excluded the same way from the existing Debian
 package golang-github-gogits-chardet, which forks this code. The tests that
 read them are skipped; see debian/rules.

Files: *
Copyright: 2012 chardet Authors
License: Expat and ICU
Comment: The upstream LICENSE grants Expat and then states: "Partial of the
 Software is derived from ICU project. See icu-license.html for license of the
 derivative portions." Upstream does not identify which files are the derived
 ones, so the union of both licences is declared for the whole tree. The
 detection tables in the recogniser sources are the portions derived from ICU.

Files: debian/*
Copyright: 2026 Juan Manuel Méndez Rey <juan.mendezr@proton.me>
License: Expat
Comment: Debian packaging is licensed under the same terms as upstream

License: Expat
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal
 in the Software without restriction, including without limitation the rights
 to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 copies of the Software, and to permit persons to whom the Software is
 furnished to do so, subject to the following conditions:
 .
 The above copyright notice and this permission notice shall be included in all
 copies or substantial portions of the Software.
 .
 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 SOFTWARE.

License: ICU
 COPYRIGHT AND PERMISSION NOTICE
 .
 Copyright (c) 1995-2012 International Business Machines Corporation and others
 All rights reserved.
 .
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal
 in the Software without restriction, including without limitation the rights
 to use, copy, modify, merge, publish, distribute, and/or sell copies of the
 Software, and to permit persons to whom the Software is furnished to do so,
 provided that the above copyright notice(s) and this permission notice appear
 in all copies of the Software and that both the above copyright notice(s) and
 this permission notice appear in supporting documentation.
 .
 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF THIRD PARTY RIGHTS.
 IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE BE
 LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES, OR ANY
 DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
 ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN
 CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
 .
 Except as contained in this notice, the name of a copyright holder shall not
 be used in advertising or otherwise to promote the sale, use or other dealings
 in this Software without prior written authorization of the copyright holder.

DFSG Assistant LLM REVIEW 0 blocker, 1 important, 2 minor

This is an LLM based review using a Debian Inference based workflow. It was not written by a member of the DFSG team and has not been checked by one. It is not a decision on the package: it may be wrong, and it may have missed things.

Reviewed 3 hours ago.

Report◂

DFSG NEW review for golang-github-saintfish-chardet version 0.0~git20230101.5e3ef4b+dfsg-1

Summary

  • Licensing is DFSG-compliant and debian/copyright is accurate: No
  • Packaging is of good quality: Yes
  • Issues: 0 blocker, 1 important, 2 minor
  • debian/watch: version 5
  • debian/copyright: machine-readable
  • Scan before the review: nothing found

Licensing

The package is DFSG-free: the only licences are Expat (LICENSE, Copyright (c) 2012 chardet Authors) and the ICU licence (icu-license.html, IBM), both free and compatible, and debian/copyright is DEP-5 with the canonical Format URI and both licence texts reproduced verbatim. What is not accurate is the copyright attribution: the single Files: * stanza names only chardet Authors, although icu-license.html carries IBM's notice and the stanza itself says the detection tables in the recogniser sources are ICU derivatives. IBM's copyright line does appear inside the License: ICU paragraph, so nothing is undistributable, but the Copyright field does not record it for the files it applies to. That is reported as the one licensing issue.

Packaging

Textbook Go-team packaging: debhelper compat 14 via dh-go (which supplies the golang addon and buildsystem), dpkg-build-api 1, a single DEP-3 patch adding the missing go.mod, DEP-5 copyright with the repack documented, XS-Go-Import-Path, Vcs-*, an autopkgtest (Testsuite: autopkgtest-pkg-go) and a current Standards-Version. Two minor defects: the dh_auto_test override passes a -run pattern that matches no test, so the whole suite is silently skipped, and debian/gbp.conf's dist = DEP14 is not a distribution. Neither breaks the build or the installed library.

Issues

1. [important] [licensing] Files: * does not name International Business Machines Corporation as holder of icu-license.html or of the ICU-derived tables

  • Files: debian/copyright:14-21, icu-license.html:14, single_byte.go:782-881
  • Basis: checked, in the package or with a reference tool
  • Evidence: Files: * / Copyright: 2012 chardet Authors / License: Expat and ICU, while the same stanza's comment says The detection tables in the recogniser sources are the portions derived from ICU.; icu-license.html:14 reads Copyright (c) 1995-2012 International Business Machines Corporation and others. The only place IBM appears is the text of the stand-alone License: ICU paragraph (debian/copyright:50-51), not a Copyright field.
  • Problem: copyright-format 1.0 section 6.8 requires a Files stanza's Copyright field to collect all relevant copyright notices for the files it matches, and Policy 12.5 requires that information in /usr/share/doc/PACKAGE/copyright. The stanza matching icu-license.html and the ICU-derived data misattributes them to chardet Authors. No licence is missing and nothing is undistributable (the IBM notice is quoted in full in the ICU paragraph), so this is an accuracy defect, not a DFSG problem.
  • Fix: Add a stanza Files: icu-license.html with Copyright: 1995-2012 International Business Machines Corporation and others and License: ICU, and add IBM's copyright to the Files: * Copyright (or in a stanza covering single_byte.go's ICU-derived tables).

2. [minor] [packaging] dh_auto_test is given -run 'TestConsumeString|TestRecognisers', which matches no test, so no test ever runs

  • Files: debian/rules:9, detector_test.go:11, example_test.go:16, debian/copyright:11-12
  • Basis: checked, in the package or with a reference tool
  • Evidence: dh_auto_test -- -short -run 'TestConsumeString|TestRecognisers'; the package's only test functions are func TestDetector(t *testing.T) and func ExampleTextDetector(). No source file defines TestConsumeString or TestRecognisers, and no test calls testing.Short().
  • Problem: Because the pattern matches nothing, go test reports testing: warning: no tests to run and exits 0: the build-time test step and the autopkgtest (dh_golang_autopkgtest runs debian/rules build) verify nothing, including the library's only regression test. The comment in debian/rules and debian/copyright that the testdata-reading tests are skipped is also wrong: nothing skips, the whole test binary is filtered out. -short has no effect.
  • Fix: Patch detector_test.go to skip when the fixture is absent (if _, err := os.Stat(filepath.Join("testdata", d.File)); err != nil { t.Skip(...) }) and drop the override, or at least use -skip TestDetector (Go 1.20+, the crate builds with golang-any) instead of a -run that matches nothing; correct the comment in debian/copyright.

3. [minor] [packaging] debian/gbp.conf sets dist = DEP14, which is not a distribution

  • Files: debian/gbp.conf:3
  • Basis: checked, in the package or with a reference tool
  • Evidence: dist = DEP14 under [DEFAULT]. gbp.conf(5): keys are the command-line options with the '--' stripped off (for gbp buildpackage, with --git- stripped); gbp-buildpackage(1) documents --git-dist=DIST as Build for distribution DIST when using git-pbuilder.
  • Problem: DEP14 is Debian's branch-naming policy, not a Debian suite, so the setting is taken as the pbuilder/cowbuilder distribution: gbp buildpackage --git-pbuilder would look for a DEP14 chroot and fail. It sets nothing about the branch layout it appears to be meant for.
  • Fix: Delete the line (gbp defaults to the DEP-14 layout already), or set a real suite such as dist = unstable.

Verified

  • Tree inventory: the orig tarball contains 13 files (2022.go, AUTHORS, detector.go, detector_test.go, example_test.go, icu-license.html, LICENSE, multi_byte.go, README.md, recognizer.go, single_byte.go, unicode.go, utf8.go); the unpacked tree adds only go.mod, which comes from debian/patches/0001-add-go-mod.patch. testdata/ is indeed absent, as Files-Excluded: testdata declares.
  • Licence evidence over the whole tree (grep for Copyright/(C)/©/SPDX/Licensed under/GNU/Apache/Mozilla/public domain/All rights reserved/IBM/ICU): no .go file carries any licence or copyright header. The only licence statements are LICENSE (Expat, Copyright (c) 2012 chardet Authors, plus the pointer to icu-license.html) and icu-license.html (ICU, IBM). AUTHORS names Sheng Yu, consistent with Upstream-Contact.
  • debian/copyright is DEP-5 (canonical Format: URI). The Expat and ICU texts are reproduced verbatim and completely; neither licence is in /usr/share/common-licenses, so including them is required. Both licences are free and compatible when combined.
  • The prescan file icu-license.html is a 1.9 KB verbatim HTML copy of the ICU 1.8.1-and-later licence, not a built, minified or binary artefact; it is IBM's text, which is why the missing holder is reported above.
  • No built binaries, bytecode, minified code, archives, PDFs, fonts or images in the tree. single_byte.go's 44.9 KB of tables (charMap_/ngrams_ at 782-881) carry no generated/DO NOT EDIT marker and are the hand-written ICU-derived data the copyright comment describes, so no generator input is missing.
  • debian/lrc.config is a legitimate licenserecon per-package config, not dead weight: lrc(1) documents debian/lrc.config, and /usr/share/lrc/lrc.config gives the syntax d/copyright name | licensecheck name. Expat and ICU | ICU maps the declared union to what licensecheck reports for icu-license.html, so it hides no real mismatch.
  • Build dependencies check out: dh-go 1.69 in unstable provides the golang debhelper addon and buildsystem (apt-cache), dpkg-build-api (= 1) is provided by dpkg-dev 1.23.11, and the lintian override for the dh-golang addon tag is justified because dh-go does not Provide dh-sequence-golang.
  • Standards-Version 4.7.4 against Policy 4.7.4.1 (policy tool) is current; dpkg-build-api 1 makes Rules-Requires-Root default to no, so its absence is correct. The ${misc:Depends}, in Depends at compat 14 is harmless.
  • The single patch has a DEP-3 header (Description, Author, Forwarded: not-needed, Last-Update), adds only go.mod with no external requires, and is listed in patches/series and applied. XS-Go-Import-Path, Homepage, Vcs-Git/Vcs-Browser, Section: golang, Architecture: all with Multi-Arch: foreign and Testsuite: autopkgtest-pkg-go are present and consistent with Go-team practice.
  • grep for func Test/func Example and for the -run names found only TestDetector and ExampleTextDetector, confirming the filter matches nothing. All files under debian/ were read (watch excepted, as it is outside this review), and every upstream file was read in full or in the relevant ranges.
  • uscan found the packaged version as the newest upstream release (upstream 0.0~git20230101.5e3ef4b, packaged 0.0~git20230101.5e3ef4b+dfsg).

Not confirmed

  • No build was run. With the -run pattern matching no test, go test is expected to print no tests to run and exit 0, so the build and the autopkgtest are assumed to pass; the test-coverage loss in issue 2 does not depend on that assumption.
  • golang-github-gogits-chardet-dev Provides: golang-github-saintfish-chardet-dev (apt-cache). I could not list its files, so whether it also installs sources under /usr/share/gocode/src/github.com/saintfish/chardet (which would collide with this package's file list) is unchecked.

Proposed changes

None.

Uploader's response to the LLM review

The uploader rejected this upload in response to the review.

Review Information

rejected — allocated to dfsg-uploader 3 hours ago, started 3 hours ago, completed 3 hours ago.

Final Comment

The uploader rejected this upload in response to the automated (LLM) review of it, so that they can fix the problems it found and upload again.

The uploader said:

All three are fixed in
https://salsa.debian.org/go-team/packages/golang-github-saintfish-chardet/-/merge_requests/7

The -run filter named two tests that do not exist. The only test is
TestDetector, and it reads the testdata that d/copyright excludes as
non-free, so it is skipped by name now and the suite runs again.

dist = DEP14 is gone, and debian-branch now says master, which is the
packaging branch of the repository.

Back to Dashboard | View all reviews for this package