sys-fs/dwarfs: A fast high compression read-only file system

I has just ported new package:
https://github.com/mhx/dwarfs
It's file-system like squashfs but compresses stronger

I had to use built-in libs because they hard-coded,
and furthermore they used partially

So It no more require dev-cpp/folly,
and I thought about deleting its.

Package-Manager: Portage-3.0.9, Repoman-3.0.2
Signed-off-by: Denis Reva <denis7774@gmail.com>
This commit is contained in:
Denis Reva
2020-12-02 12:34:35 +05:00
parent 84d883e2b2
commit c54f1d5636
3 changed files with 115 additions and 0 deletions

3
sys-fs/dwarfs/Manifest Normal file
View File

@@ -0,0 +1,3 @@
DIST dwarfs-0.2.3.tar.gz 131021 BLAKE2B 4bdcbc7c775b4e20dc523f77e7fb84a9cd978df8c63bfd2300343bbf4dd49394e8f0fb1779ac990174c38acb2dbe1444404c518c22382da8e36e13e65343c568 SHA512 4638f3a9d712ead6701f3b739d0239ddf5f742c02cf4723079df702549c3af179b798b833bd69b77eeffb68df670fbcedff70ef8d7a88e4c9e5dd42b090eed63
DIST fbthrift-2020.11.30.00.tar.gz 5274706 BLAKE2B ad13eb7b66f804c7a667746d72298e05726d1f78a49523a10f2712cc4dbea71999590a2c420e8081d32dc24a7d203e11d5c7e5c15f2627594295cd5533096dd9 SHA512 33b4c1a4cfc8783f99d5da8ae113f877e1f23586f5a854833107805fe35834604e677d7abe690cefeb70baed20cb3cc14449cfae35a9af89594974377bbe0329
DIST folly-2020.11.30.00.tar.gz 3261437 BLAKE2B 5070819f1f8bac2030e761a8d2330379e8e5ba242d21330b98dcd598cb18e12046a9ad3fb6462f82dbc47c53cf0905c330e707201bfbbedb81f203ef1b30731e SHA512 e7e06bd8c8e38af345d20a04886b4497c4f0c9877263dbcb5a3f2d17c7a3252a2529086a5a04d07c2d3a457ca3930c43672fc4f74b7bdbc2ec0d07abdf1ed425

View File

@@ -0,0 +1,93 @@
# Copyright 1999-2020 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2
EAPI=7
inherit cmake check-reqs
DESCRIPTION="A fast very high compression read-only FUSE file system"
HOMEPAGE="https://github.com/mhx/dwarfs"
FBTHRIFT_MAGIC="2020.11.30.00"
FOLLY_MAGIC="2020.11.30.00"
SRC_URI="https://github.com/mhx/dwarfs/archive/v${PV}.tar.gz -> ${P}.tar.gz
https://github.com/facebook/fbthrift/archive/v${FBTHRIFT_MAGIC}.tar.gz -> fbthrift-${FBTHRIFT_MAGIC}.tar.gz
https://github.com/facebook/folly/archive/v${FOLLY_MAGIC}.tar.gz -> folly-${FOLLY_MAGIC}.tar.gz"
LICENSE="GPL-3"
SLOT="0"
KEYWORDS="~amd64 ~x86"
IUSE="jemalloc"
DEPEND="app-arch/zstd
app-arch/lz4
app-arch/xz-utils
app-arch/snappy
dev-libs/boost[context,threads]
dev-libs/double-conversion
dev-libs/libfmt
dev-libs/libevent
dev-cpp/gflags
dev-cpp/glog[gflags]
dev-cpp/sparsehash
sys-libs/binutils-libs
sys-libs/libunwind
sys-fs/fuse:3
sys-devel/flex
sys-devel/binutils:*
sys-libs/zlib
jemalloc? ( dev-libs/jemalloc )"
RDEPEND="dev-libs/boost[context,threads]
dev-libs/double-conversion
app-arch/zstd
app-arch/lz4
app-arch/xz-utils
app-arch/snappy
sys-fs/fuse:3
sys-libs/binutils-libs
sys-libs/libunwind"
BDEPEND="app-text/ronn
dev-util/cmake
sys-apps/sed
sys-devel/bison
virtual/pkgconfig"
CHECKREQS_DISK_BUILD="512M"
DOCS=( "README.md" "CHANGES.md" "TODO" )
src_unpack(){
default
rm -d "${S}"/fbthrift/
rm -d "${S}"/folly/
mv "${WORKDIR}/fbthrift-${FBTHRIFT_MAGIC}" "${S}"/fbthrift/
mv "${WORKDIR}/folly-${FBTHRIFT_MAGIC}" "${S}"/folly/
}
src_prepare(){
einfo "setting path to $(get_libdir)"
pushd "${S}"/folly
sed "s/lib CACHE/$(get_libdir) CACHE/" -i CMakeLists.txt || die
sed "s/lib\/cmake\/folly CACHE/$(get_libdir)\/cmake\/folly CACHE/" -i CMakeLists.txt || die
popd
pushd "${S}"/fbthrift
sed "s/lib CACHE/$(get_libdir) CACHE/" -i CMakeLists.txt || die
sed "s/lib\/cmake\/fbthrift CACHE/$(get_libdir)\/cmake\/fbthrift CACHE/" -i CMakeLists.txt || die
popd
sed "s/DESTINATION lib/DESTINATION $(get_libdir)/" -i CMakeLists.txt || die
cmake_src_prepare
}
pkg_postinst(){
elog "Test shows that dwarfs compiled with Clang is substantially faster than GCC ones"
elog "See ${HOMEPAGE}"
elog "So you may want to compile it independently with Clang by the"
elog "https://wiki.gentoo.org/wiki/Clang"
elog "And with the per-package settings:"
elog "https://wiki.gentoo.org/wiki/Handbook:AMD64/Portage/Advanced#Per-package_environment_variables"
elog "Also you may find more information in the"
elog "${HOMEPAGE}"
elog "About creating: ${HOMEPAGE}/blob/main/doc/mkdwarfs.md"
elog "About mounting: ${HOMEPAGE}/blob/main/doc/dwarfs.md"
}

View File

@@ -0,0 +1,19 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE pkgmetadata SYSTEM 'http://www.gentoo.org/dtd/metadata.dtd'>
<pkgmetadata>
<maintainer type="person">
<email>denis7774@gmail.com</email>
<name>Denis Reva</name>
<description>Partial maintainer</description>
</maintainer>
<longdescription lang="en">
DwarFS is a read-only file system with a focus on achieving very high compression ratios in particular for very redundant data.
This probably doesn't sound very exciting, because if it's redundant, it should compress well. However, I found that other read-only, compressed file systems don't do a very good job at making use of this redundancy. See here for a comparison with other compressed file systems.
DwarFS also doesn't compromise on speed and for my use cases I've found it to be on par with or perform better than SquashFS. For my primary use case, DwarFS compression is an order of magnitude better than SquashFS compression, it's 4 times faster to build the file system, it's typically faster to access files on DwarFS and it uses less CPU resources.
Distinct features of DwarFS are:
* Clustering of files by similarity using a similarity hash function. This makes it easier to exploit the redundancy across file boundaries.
* Segmentation analysis across file system blocks in order to reduce the size of the uncompressed file system. This saves memory when using the compressed file system and thus potentially allows for higher cache hit rates as more data can be kept in the cache.
* Highly multi-threaded implementation. Both the file system creation tool as well as the FUSE driver are able to make good use of the many cores of your system.
* Optional experimental Lua support to provide custom filtering and ordering functionality.
</longdescription>
</pkgmetadata>