‎MinIO-创建分布式s3对象存储集群

mtain 2022年05月13日 24次浏览

一、说明

‎MinIO:https://min.io/

‎MinIO 提供高性能、S3 兼容的对象存储。‎

二、安装

1. 环境说明

主机清单

主机名IP地址备注
s3-n01192.168.1.101
s3-n02192.168.1.102
s3-n03192.168.1.103进群负载均衡

文件目录

路径用途
/opt/minio/miniominio执行程序
/opt/minio/minio.confminio配置文件
/data1和/data2数据存储路径-最好挂载两块数据盘

2. 安装

初始化目录

useradd minio
mkdir /data1
mkdir /data2
mkdir /opt/minio
chown -R minio:minio /data1
chown -R minio:minio /data2
chown -R minio:minio /opt/minio

**注意:**一机两个数据磁盘,data1和data2是多磁盘路径

上传minio

上传minio到/opt/minio中,并增加可执行权限chmod +x /opt/minio/minio

创建minio配置文件

cat <<EOT >> /opt/minio/minio.conf
MINIO_OPTS="--console-address :9001"
MINIO_ROOT_USER=root
MINIO_ROOT_PASSWORD=MVR62XkSuj1d
MINIO_VOLUMES="http://192.168.1.101/data1 http://192.168.1.101/data2 http://192.168.1.102/data1 http://192.168.1.102/data2"
EOT

集群节点数量必须是4的倍数

创建服务文件

vi /etc/systemd/system/minio.service

[Unit]
Description=MinIO
Documentation=https://docs.min.io
Wants=network-online.target
After=network-online.target
AssertFileIsExecutable=/opt/minio/minio

[Service]
WorkingDirectory=/opt/minio

User=minio
Group=minio
ProtectProc=invisible
PermissionsStartOnly=true
EnvironmentFile=-/opt/minio/minio.conf
ExecStartPre=/bin/bash -c "if [ -z \"${MINIO_VOLUMES}\" ]; then echo \"Variable MINIO_VOLUMES not set in /opt/minio/minio.conf\"; exit 1; fi"
ExecStart=/opt/minio/minio server $MINIO_OPTS $MINIO_VOLUMES

# Let systemd restart this service always
Restart=always

# Specifies the maximum file descriptor number that can be opened by this process
LimitNOFILE=65536

# Specifies the maximum number of threads this process can create
TasksMax=infinity

# Disable timeout logic and wait until process is stopped
TimeoutStopSec=infinity
SendSIGKILL=no

[Install]
WantedBy=multi-user.target

# Built for ${project.name}-${project.version} (${project.name})


服务启动

# 刷新服务配置
systemctl daemon-reload

systemctl start minio
systemctl enable minio

systemctl status minio

nginx配置

upstream http_minio {
    server 192.168.1.101:9000;
    server 192.168.1.102:9000;
}

server{
    listen       9000;
    server_name  localhost;

    ignore_invalid_headers off;
    client_max_body_size 0;
    proxy_buffering off;

    location / {
        proxy_set_header   X-Real-IP $remote_addr;
        proxy_set_header   X-Forwarded-Host  $host:$server_port;
        proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Proto  $http_x_forwarded_proto;
        proxy_set_header   Host $http_host;

        proxy_connect_timeout 300;
        proxy_http_version 1.1;
        chunked_transfer_encoding off;
        proxy_ignore_client_abort on;

        proxy_pass http://http_minio;
    }
}

3. 访问

文件操作:http://192.168.1.103:9000
管理端:http://192.168.1.101:9001

三、多租户权限设置

需求: 某用户只能访问自己的bucket(或者可以做成:访问以自己用户名开头的bucket)

实现方式: Policy权限设置,非常灵活的权限设置方式

设置步骤

  1. 创建Policy,其中指定bucket前缀
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                    "s3:DeleteObject",
                    "s3:GetObject",
                    "s3:ListBucket",
                    "s3:PutObject"
                ],
                "Resource": [
                    "arn:aws:s3:::【用户名】*"
                ]
            }
        ]
    }
    
  2. 创建用户【用户名】,授权新建的Policies
  3. 进入指定用户,创建Service Accounts,最后创建Bucket