Azure Storage (Azure儲存體服務)
Storage Account
必須先建立Storage Account然後才能建立Azure Storage，同一個Storage Account可建立多個不同種類的Storage
Performance:
- Standard : optimized for high capacity and high throughput 通常選用此項
- Premium : optimized for high transaction rates and single-digit consistent storage latency 通常用於VM Disk
Performance tiers for block blob storage
Account Kind
Replication:
Standard-Replication
Premium-Replication
Locally redundant storage (LRS): 只在一個Data Center中複製3份
Zone-redundant storage (ZRS):
只在一個region最少2個最多3個資料中心，總共也只有3份
需選StorageV2 (general purpose V2)
PS.但Premium不支援Zone-redundant
Live migration to ZRS from Azure support is supported only for storage accounts that use LRS or GRS replication.
Geo-redundant storage (GRS): primary region 3份，region pair也留3份 (備用region平時不可用)
Read access Geo-redundant storage(RA-GRS): 最貴，備用可以讀
Geo-Zone-redundant storage (GZRS): 資料複製跨3個資料中心並且複製到region pair，共6份
Read-access Geo-zone-redundant storage(RA-GZRS): 資料複製跨3個資料中心並且複製到region pair，共6份，region pair資料可讀
General-purpose V2支援的功能較多且費用較便宜
General-purpose V2才支援Storage Account存取層級
影響Blob
預設Access Tier(File上傳時的預設選項)
- Hot - Optimized for storing data that is accessed frequently.
- Cool - Optimized for storing data that is infrequently accessed and stored for at least 30 days. (不常存取，但立即取得且成本較Hot低)
- Archive - Optimized for storing data that is rarely accessed and stored for at least 180 days with flexible latency requirements (on the order of hours). (若要讀取檔案，可修改成其他Tier例如Hot or Cool或Copy到其他Tier)。Rehydrate blob data from the archive tier https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blob-rehydration?tabs=azure-portal
Azure Blob storage: hot, cool, and archive access tiers
上傳檔案時可以指定不同Access tier
Blob Lifecycle Management
Storage accounts>YourStorageAccount> Lifecycle Management
依照規則搬移調整到不同access tier，例如將超過30天的檔案搬移切換到Cool，超過180切換到Archive
Storage Service Encryption(SSE)
All Storage account are encrypted using Storage Service Encryption(SSE) for data at rest (靜止/待用狀態) 除了傳輸過程以外，資料在靜止/待用狀態都是加密的。
Customer-managed keys
Storage Type
1.Azure Blob
2.Azure Files (accessible via SMB protocol 445/TCP, 取代file server)
4.Azure Tables (儲存半結構化NoSQL，Key-Value格式，例如JSON,XML，另外有一個Azure Cosmos DB服務也可儲存半結構化NoSQL資料)
有勾選啟用 DATA LAKE STORAGE GEN2
Azure Storage Documentation
https://docs.microsoft.com/en-us/azure/storage/
Data Lake Gen2 file systems : 新增時有勾選啟用 DATA LAKE STORAGE GEN2
Azure Blobs (objects) : 新增時無勾選啟用 DATA LAKE STORAGE GEN2
用於儲存非結構化資料 (Text/CSV file, Image file, Video files, Audio file)
Commons uses:
Commons uses:
- saving images or documents directly to a browser.
- Storing files for distributed access, such as installation.
- Streaming video and audio.
- Storing data for Backup and restore, disaster recovery, and archiving.
- Storing data for analysis by an on-promises or Azure-hosted service.
2.Azure Files (accessible via SMB protocol 445/TCP, 取代file server)
用於儲存非結構化資料 (Text/CSV file, Image file, Video files, Audio file)
3.Azure Queues (儲存large numbers of messages)
4.Azure Tables (儲存半結構化NoSQL，Key-Value格式，例如JSON,XML，另外有一個Azure Cosmos DB服務也可儲存半結構化NoSQL資料)
有勾選啟用 DATA LAKE STORAGE GEN2
未勾選啟用 DATA LAKE STORAGE GEN2
除了Azure Portal還可使用Azure Storage Explorer管理(檢視與異動資料)
若Storage Account建立時有勾選 啟用 DATA LAKE STORAGE GEN2
在Storage Explorer的總管視窗就會在Storage Account後面多出(ADLS Gen2)
Storage的使用方式
Blob Containers
使用Blob必須先建立Blob Container
- Organization : 分組集合，如同目錄
- Public Access Level : 公開存取層級
- Access Policy : 存取原則
存取方式
1.透過Azure Storage Explorer
下載安裝Azure Storage Explorer
連接到Azure之後，在Storage Account建立Container
fs1與 fs2 是一個Container(類似一個邏輯目錄)
fs1裡面才作為存放檔案用
2.Azure Portal
可建立Container並可上傳檔案，可選多個檔案進行多線同時上傳
Azure Blobs的3種格式
1.Block Blobs (用於text or binary files, and for uploading large files efficiently) 預設Portal上傳檔案格式(進階選項可以看到格式)
2.Append Blobs (made up of blocks, but they are optimized for append operations, making them ideal for logging scenarios)
3.Page Blobs (用於Azure VM Disk(vhd檔), Azure SQL DB, 可使用REST APIs存取 )
3.檔案層級
(1)針對每個檔案可以產生各自的SAS
Permission: Read/Create/Write/Delete
然後使用https URL直接取得檔案
(2)Container層級
有三種存取層級Access Level
When public access is allowed for a storage account, you can configure a container with the following permissions:
- No public read access: The container and its blobs can be accessed only with an authorized request. This option is the default for all new containers. (Private(no anonymous access)) 無匿名存取，例如透過SAS產生的URL https://yourStorageAccountName.blob.core.windows.net/yourContainerName/YourFileName.wav?sp=r&st=2020-08-14T17:10:50Z&se=2020-08-15T01:10:50Z&spr=https&sv=2019-12-12&sr=b&sig=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 取得檔案
- Public read access for blobs only: Blobs within the container can be read by anonymous request, but container data is not available anonymously. Anonymous clients cannot enumerate the blobs within the container. (Blob(anonymous read access for blobs only)) 可以匿名存取，但無法列出Container裡面的檔案清單，例如透過 https://yourStorageAccountName.blob.core.windows.net/yourContainerName/YourFileName.wav 取得檔案
- Public read access for container and its blobs: Container and blob data can be read by anonymous request, except for container permission settings and container metadata. Clients can enumerate blobs within the container by anonymous request, but cannot enumerate containers within the storage account. (Container (anonymous read access for containers and blobs)) 可以匿名取得檔案，也可以列出Container裡面的檔案清單
Configure anonymous public read access for containers and blobs
Access Policy控制存取
Data protection
Turn on soft delete for blobs (預設刪除的檔案可保留7天)
Azure Table
以下方為例，有新增一個JTable，按下Add輸入資料時可自訂義欄位ProductionName與ProductionType
Azure Storage Documentation
https://docs.microsoft.com/en-us/azure/storage/
控管存取網路
Storage account> firewall and virtual networks
Allow access from 將All networks 改成 selected networks> Add existing virtual network
也可以指定允許存取的IP address
AzCopy
Get started with AzCopy
Storage type Currently supported method of authorization
Blob storage Azure AD & SAS
Blob storage Azure AD & SAS
(hierarchical namespace)
File storage SAS only
ex. copy the contents of D:\Folder1 to the public container in an Azure Storage account named contosodata. 此題URL後面是Container，所以type是blob
azcopy copy D:\folder1 https://contosodata.blob.core.windows.net/public --recursive
Azure Storage Explorer
Support access by shared access signatures (SAS)
Mounting network drive
Only support by storage account name and storage account key
Use an Azure file share with Windows
UNC path format is \\<storageAccountName>.file.core.windows.net\<fileShareName>.
For example: \\anexampleaccountname.file.core.windows.net\example-share-name.
Do not support by SAS
SAS key for UNC Path
Azure Container instances
What is Azure Container Instances?
Persistent storage
To retrieve and persist state with Azure Container Instances, we offer direct mounting of Azure Files shares backed by Azure Storage.
Exercise - Use data volumes
mount an Azure file share to an Azure container instance so you can store data and access it later
Azure Kubernetes Services (AKS)
Deploy a SQL Server container in Kubernetes with Azure Kubernetes Services (AKS)
configure a SQL Server instance on Kubernetes in Azure Kubernetes Service (AKS), with persistent storage
Azure Disk
Azure Import/Export service
Azure Import/Export system requirements
Supported storage accounts
Azure Import/Export service supports the following types of storage accounts:
- Standard General Purpose v2 storage accounts (recommended for most scenarios)
- Blob Storage accounts
- General Purpose v1 storage accounts (both Classic or Azure Resource Manager deployments)
Supported storage types
The following list of storage types is supported with Azure Import/Export service.
匯出Export只支援Blob
匯入Import則支援Blob與File
Azure File Sync
Sync group
- A sync group must contain one cloud endpoint, which represents an Azure file share and one or more server endpoints. 只能有一個Cloud endpoint，可以有多個Server endpoint
- A server endpoint represents a path on a registered server.
- A server can have server endpoints in multiple sync groups. 同一Server的server endpoint可以在多個sync group
Deploy Azure File Sync
Cloud Tiering:
When enabled, cloud tiering will tier files to your Azure file shares. This converts on-premises file shares into a cache, rather than a complete copy of the dataset, to help you manage space efficiency on your server. With cloud tiering, infrequently used or accessed files can be tiered to Azure Files.
Cloud Tiering Overview
沒有留言:
張貼留言