NPW Insights (Free): Week 3 of 4 for Solution Architect

Data Management

Key AI and data trends in 2023: Google and IDC report

About report: Surveys 800 global organizations on data and AI trends.
Adoption themes: Open source technologies to prevent vendor lock-in; unified data clouds to improve data usage and governance, and formation of open data ecosystems.
Key recommendations: Eliminate siloed DBs and warehousing strategies as they lead to conflicting insights; teams must get to know unknown data to understand security risk exposure. Integrate insights into the workflows of users instead of delivering them within BI.

AWS Glue Crawlers now integrate with AWS Lake Formation

• With the integration, Glue Crawler can access S3 targets when Lake Formation is used to manage them, with Lake Formation permissions.
• Also supports integration with Lake Formation for a different account.

Also: Now implement RBAC in Amazon QuickSight for data sources that connect to Amazon S3 and Amazon Athena.

Architecture Design

Azure publishes reliable web app pattern for .NET

• Based on Azure Well-Architected Framework, it provides guidance and reference architecture to transform traditional .NET apps to modern apps.
•  See the reference architecture that applies the reliable web app pattern, and business and technical objectives that are achieved by it.

Microservices architecture evolving to serverless monoliths

• Frontend frameworks and Backend for Frontend (BFF) frameworks are converging into front-end meta-frameworks like Next.js, and Remix.
• Backend logic can now be handled by a single DBaaS or BaaS technologies like Firebase.
• Converging frontend and backend stacks along with mono-repository architectures, enable applications powered by just 2 technologies in the stack.

The future of WebAssembly (Wasm)

• WASI, system interface for WebAssembly, extending the reach of Wasm beyond browsers
• Wasm yet to achieve the requisite maturity for backend apps
• Kubernetes and Wasm to grow solve in tandem, with latter solving problems related to application runtime

AI/ML

Document AI Workbench updated and now generally available

• Now automatically detects and creates existing schema labels from pre-annotated documents.
• Processor Gallery recommends best models for the use-case; Labeling UI improved.
• Upcoming features: Custom Document Classifier, support for larger documents and non-Latin languages

MLOps best practices with Amazon SageMaker for real-time inference models

Scope: How to combine SageMaker model variants - production and shadow - and deployment guardrails with MLOps tools to generate end-to-end patterns for model development. Also included is example execution for canary and linear shifting deployment guardrails.
What’s covered: Model development starting with SageMaker pipelines (data preparation, model artifacts and metadata storage), moving on to production testing with A/B or shadow deployment. Then, rollout with a SageMaker endpoint using a deployment guardrail, and finally monitoring for drift in model and data quality.

Security

Highlights of CloudNativeSecurityCon 2023

Software supply chain security: A new project that aggregates software supply chain data; approaches to build trust between software supply chain artifacts; emerging Supply Chain Levels for Software Artifacts (SLSA) standard that maps relationships between artifacts.
Spotlight: Yahoo demonstrated its approach to software supply chain security across 700+ K8s clusters – image signature and freshness check policy was a highlight, as they publish 5K+ container images daily.
Other themes: Using IaC for automating policy-based compliance; current state of software supply chain verification capabilities and projects like Kyverno, GUAC, and Sigstore policy controller.

Cloud security tips for financial services

• Translate regulatory requirements to cloud security strategy, and adopt tailored approaches to securing cloud data
• Leverage agentless security strategy to deploy tools at scale and unify security for legacy and cloud environments.

Cost Management

2 new AWS Cost Categories features for grouping resources

• Now group AWS resources by Region, and use the OR operator to define cost categories rules.
• With ‘OR’, rules can now be more inclusive across dimensions (Linked Account, Charge Type, Service, Cost Allocation Tags, Region, Cost Category).

Tagging strategy for granular cloud cost visibility in AWS

What it does: Requires creating  tag taxonomy, documenting tagging strategy (to attribute spend to cost centers), and enforcing it across teams.
What’s covered: How to create tag policies for tags in AWS Organizations (top-down or child organizations driven); attach policy to organizational units to enforce them across organization; and use of Service Control Policies for stricter enforcement.
Also: Use Tag Editor to identify untagged resources, and AWS Config to support ongoing compliance.

Also: Ruby on Rails creator and Basecamp and HEY co-founder shares updates on how cloud exit will save them $7m over the next 5 years.

Career

Red Hat expert-speak: how the role of the Solution Architect is evolving

• Modern tooling has trivialized traditional roles of the architect such as ensuring scalability;
• Continuous delivery means architecture is constantly evolving
• Software teams, with telemetry data, now better aware of how customers interact with services, which enables them to respond to their needs.
• These shifts have made architecture a team sport rather than an individual’s responsibility; architects must become enablers and mentors.

Also: AWS Gallup APAC Digital Skills Report released. Read here.

Provisioning + Runtime

SQL Server on Azure VMs better price-performance than on EC2: GigaOm report

• Runs 57% faster and costs 54% less than on EC2 with 3-year commitment and Azure Hybrid Benefit (study commissioned by Microsoft)
• Azure Ebdsv5 VMS optimized for database workloads, and Premium SSD v2 Disk Storage

Azure Managed Lustre enters public preview

• Lustre is an open-source parallel file system for large-scale cluster computing, ideal for HPC and AI workloads.
• Built on Azure Managed Disks, two SSD-based SKUs will be offered with 125MBps and 250MBps per TB of capacity, scalable up to 768TBs.

Also: Azure HPC Cache Premium Read-Write, which provides up to 84TB capacity for a single cache and 20GBps read throughput at low latency is now in preview; Azure HPC Cache – Standard price dropped.

Trending topics last week

• Provisioning related news accounted for 45% of total attention on stories
• App building related updates accounted for another 27%

CSP trends last week

• Databases saw the most important announcements, both from Azure.
• In fact, Azure updates accounted for 52% of total attention on stories
• Google Cloud and Azure had important updates in VMs, the second most active topic

Products that trended last week

• Caching becoming possible in Azure Container Registry instance
• Confidential GKE Nodes' availability in confidential VMs
• Azure SQL updates including automatic key rotation for CMKs
• Azure Cache for Redis allowing enhanced passive geo-replication
• Stateful firewall rules to tag-based resources in AWS Firewall Network