Ai大模型相关案例,AI
Alluxio 1.2.0 发布了，Alluxio 是一个高容错的分布式文件系统，允许文件以内存的速度在集群框架中进行可靠的共享，类似Spark和 MapReduce。通过利用lineage信息，积极地使用内存，Alluxio的吞吐量要比HDFS高300多倍。Alluxio都是在内存中处理缓 存文件，并且让不同的 Jobs/Queries以及框架都能内存的速度来访问缓存文件。
改进记录如下：
- Richer feature set and improved performance for S3 under storage:
- Support for Amazon’s official s3 client library, aws-sdk through the
s3a://scheme. - S3a achieves up to 4x higher write throughput compared to s3n.
- S3a achieves up to 2x higher metadata throughput compared to s3n.
- Support for advanced features including instance profile login, server side encryption, and secure http protocol.
- Jets3t client library is still available under the
s3n://scheme.
- Support for Amazon’s official s3 client library, aws-sdk through the
- Improved performance for Swift under storage: Alluxio 1.2 streamlines operations to Swift under storages, cutting down the number of requests to Swift per command by up to 4x compared to previous versions of Alluxio.
- Transparent ACLs between Alluxio and under storages with ACL support:Automatically propagates user, group, and permissions information from Alluxio to the under storage. Automatically fetches user, group, and permissions information from the under storage into Alluxio when new file metadata are discovered. Enable this feature by setting the flag
alluxio.security.authorization.permission.enabledto true. - Improved file system metadata management: Alluxio 1.2 manages metadata over 3x more effectively compared to Alluxio 1.1.1, allowing users to store and access triple the amount of metadata compared to before.
- More flexible under storage metadata management: Alluxio now provides finer grained control over metadata loaded from the under storage. For each operation which may need to consult the under storage, users can specify if they want to always sync new data, only sync new data on the first access, or never sync new data.
下载地址：http://www.alluxio.org/download
相关文章
