Expanded from Tyler Akidau’s popular blog posts "Streaming 101" and "Streaming 102", this book takes you from an introductory level to a nuanced understanding of the what, where, when, and how of processing real-time data streams. You’ll also dive deep into watermarks and exactly-once processing with co-authors Slava Chernyak and Reuven Lax.
Tyler Akidau is a senior staff software engineer at Google, where he is the technical lead for the Data Processing Languages & Systems group, responsible for Google's Apache Beam efforts, Google Cloud Dataflow, and internal data processing tools like Google Flume, MapReduce, and MillWheel. His also a founding member of the Apache Beam PMC. Though deeply passionate and vocal about the capabilities and importance of stream processing, he is also a firm believer in batch and streaming as two sides of the same coin, with the real endgame for data processing systems the seamless merging between the two. He is the author of the 2015 Dataflow Model paper and the Streaming 101 and Streaming 102 articles on the O’Reilly website. His preferred mode of transportation is by cargo bike, with his two young daughters in tow.
Slava Chernyak is a senior software engineer at Google Seattle. Slava spent over five years working on Google’s internal massive-scale streaming data processing systems and has since become involved with designing and building Windmill, Google Cloud Dataflow's next-generation streaming backend, from the ground up. Slava is passionate about making massive-scale stream processing available and useful to a broader audience. When he is not working on streaming systems, Slava is out enjoying the natural beauty of the Pacific Northwest.
Reuven Lax is a senior staff software engineer at Google Seattle, and has spent the past nine years helping to shape Google's data processing and analysis strategy. For much of that time he has focused on Google's low-latency, streaming data processing efforts, first as a long-time member and lead of the MillWheel team, and more recently founding and leading the team responsible for Windmill, the next-generation stream processing engine powering Google Cloud Dataflow. He's very excited to bring Google's data-processing experience to the world at large, and proud to have been a part of publishing both the MillWheel paper in 2013 and the Dataflow Model paper in 2015. When not at work, Reuven enjoys swing dancing, rock climbing, and exploring new parts of the world.
In this collection of over 80 columns, Eric Brechner's alter ego pulls no punches with his candid commentary and best practice solutions to the issues that irk him the most. He dissects the development process, examines tough team issues, and critiques how the software business is run, with the added touch of clever humor and sardonic wit. His ideas aren't always popular (not that he cares), but they do stimulate discussion and imagination needed to drive software excellence.
Get the unvarnished truth on how to:Improve software quality and value—from design to security Realistically manage project schedules, risks, and specs Trim the fat from common development inefficiencies Apply process improvement methods—without being an inflexible fanatic Drive your own successful, satisfying career Don't be a dictator—develop and manage a thriving team!
Companion Web site includes:Agile process documents Checklists, templates, and other resources
New chapters are included on specifying data requirements, writing high-quality functional requirements, and requirements reuse. Considerable depth has been added on business requirements, elicitation techniques, and nonfunctional requirements. In addition, new chapters recommend effective requirements practices for various special project situations, including enhancement and replacement, packaged solutions, outsourced, business process automation, analytics and reporting, and embedded and other real-time systems projects.
ReCaS stands for "Rete di Calcolo per SuperB e altre applicazioni" and is a computing network infrastructure in Southern Italy devoted to scientific and non-scientific applications within the vision of a common European infrastructure for computing, storage and network. The ReCaS project is part of the 2007–2013 European Union strategy, and was funded by the Italian Ministry of Research and Education (MIUR) for the development and enhancement of a distributed computing infrastructure of the Grid/Cloud type over the four EU 'Convergence' regions in Southern Italy: Campania, Puglia and Sicily and Calabria.
The network will be open and accessible to all researchers, public and private, and will be characterized by unprecedented computing power and storage capacity. Posted in the European Grid Infrastructure EGI, ReCaS is also an opportunity to the countries of the Mediterranean area and extends the potential of the current network.
Authors Kelsey Hightower, Brendan Burns, and Joe Beda—who’ve worked on Kubernetes at Google and other organizatons—explain how this system fits into the lifecycle of a distributed application. You will learn how to use tools and APIs to automate scalable distributed systems, whether it is for online services, machine-learning applications, or a cluster of Raspberry Pi computers.Explore the distributed system challenges that Kubernetes addressesDive into containerized application development, using containers such as DockerCreate and run containers on Kubernetes, using the docker image format and container runtimeExplore specialized objects essential for running applications in productionReliably roll out new software versions without downtime or errorsGet examples of how to develop and deploy real-world applications in Kubernetes
Gruntwork co-founder Yevgeniy (Jim) Brikman walks you through dozens of code examples that demonstrate how to use Terraform’s simple, declarative programming language to deploy and manage infrastructure with just a few commands. Whether you’re a novice developer, aspiring DevOps engineer, or veteran sysadmin, this book will take you from Terraform basics to running a full tech stack capable of supporting a massive amount of traffic and a large team of developers.Compare Terraform to other IAC tools, such as Chef, Puppet, Ansible, and Salt StackUse Terraform to deploy server clusters, load balancers, and databasesLearn how Terraform manages the state of your infrastructure and how it impacts file layout, isolation, and lockingCreate reusable infrastructure with Terraform modulesTry out advanced Terraform syntax to implement loops, if-statements, and zero-downtime deploymentUse Terraform as a team, including best practices for writing, testing, and versioning Terraform code
Microservice technologies are moving quickly. Author Sam Newman provides you with a firm grounding in the concepts while diving into current solutions for modeling, integrating, testing, deploying, and monitoring your own autonomous services. You’ll follow a fictional company throughout the book to learn how building a microservice architecture affects a single domain.Discover how microservices allow you to align your system design with your organization’s goalsLearn options for integrating a service with the rest of your systemTake an incremental approach when splitting monolithic codebasesDeploy individual microservices through continuous integrationExamine the complexities of testing and monitoring distributed servicesManage security with user-to-service and service-to-service modelsUnderstand the challenges of scaling microservice architectures
In this collection of essays and articles, key members of Google’s Site Reliability Team explain how and why their commitment to the entire lifecycle has enabled the company to successfully build, deploy, monitor, and maintain some of the largest software systems in the world. You’ll learn the principles and practices that enable Google engineers to make systems more scalable, reliable, and efficient—lessons directly applicable to your organization.
This book is divided into four sections:Introduction—Learn what site reliability engineering is and why it differs from conventional IT industry practicesPrinciples—Examine the patterns, behaviors, and areas of concern that influence the work of a site reliability engineer (SRE)Practices—Understand the theory and practice of an SRE’s day-to-day work: building and operating large distributed computing systemsManagement—Explore Google's best practices for training, communication, and meetings that your organization can use
In three parts, this book explains how these services work and what it means to build an application the Microservices Way. You’ll explore a design-based approach to microservice architecture with guidance for implementing various elements. And you’ll get a set of recipes and practices for meeting practical, organizational, and cultural challenges to microservice adoption.Learn how microservices can help you drive business objectivesExamine the principles, practices, and culture that define microservice architecturesExplore a model for creating complex systems and a design process for building a microservice architectureLearn the fundamental design concepts for individual microservicesDelve into the operational elements of a microservices architecture, including containers and service discoveryDiscover how to handle the challenges of introducing microservice architecture in your organization