What's the problem?
If we can make it easier to make use of the information we capture as part of our architecture processes, we will help solve one of the key problems with software architecture, largely that it tends to be out of date by the time the first line of code is written.
As developers, we have many demands on our time for new features, bug requests and actually ensuring our software is usable. Even where we are able to embed coherent architecture processes, they tend to be parallel to the actual task of developing software, rather than integrated into it.
Over the last few years, a key tool in my architects toolkit has been C4, which is great for removing the requirement for software architects to be artists. It is also what sparked the idea for encapsulating all of our architectural information as code.
C4 as a knowledge base
One of the things I've been doing with C4 is using it to drive more than just diagram generation, using it as a structure to capture additional characteristics of systems and even as the foundation for generating architecture web portals to make the information more accessible to non-developers.
Through my experiments with ChatGPT over the last few months, one of the things that has become abundantly clear is that for it to really be effective you need to have clear, concise and well structured information. This is where C4 really shines, as it is designed to be a simple, structured way of capturing the key information about a system.
So, I wondered, could we make the information in our C4 models more easily accessible and able to answer questions people have about how our systems operate, thereby removing the need for developers to answer questions that require technical expertise.
Enter ArchGPT
In the FINOS Labs we have a sample trading application called TraderX. TraderX conveniently has a C4 model, so I decided to see if I could use it to generate a knowledge base that could answer questions about the system.
The first thing I needed to do was let ChatGPT understand the context I expected it to operate in.
Setting the context
You are an Enterprise Architect, we're going to look at a trading system based on it's C4 architecture. To start can you summarise the C4 architecture and how the Structurizr DSL works.
ChatGPT shows its knowledge
ChatGPT had no problem explaining the C4 model and the Structurizr DSL
Sure, I'd be happy to provide a summary of the C4 model and the Structurizr DSL.
C4 Model
The C4 Model is a framework for visualizing the architecture of a software system. It was developed by Simon Brown and stands for Context, Container, Component, and Code. The model is based on the idea of drilling down into an architecture, starting from the highest level (Context) and proceeding down to the most granulated level (Code).
- Context - At the context level, the system is viewed as a black box, and its interactions with users and other systems are outlined.
- Container - This level zooms into the system, and we start to see the high-level technology choices, how responsibilities have been divided among them, and how they interact.
- Component - Going further, the container level is broken down into smaller components, providing more detail into each module's functionality and their interactions.
- Code - This is the lowest level of the diagram where the components are broken down into classes, interfaces, or other code elements.
Structurizr DSL
Structurizr DSL (Domain-Specific Language) is a text-based language that allows developers to create software architecture diagrams using code. This can be used to define views according to the C4 model.
Using the Structurizr DSL, you can define your software systems, containers, components, and relationships between them in a structured text format. This code can be parsed and visualized in tools like Structurizr, providing an easy way to create, version control, and share architecture diagrams.
To create a Structurizr DSL, you'd have to:
- Define your workspace (including the name, description, and version) which is the wrapper for all of your model and views.
- Define your model elements (people, software systems, containers, components, relationships).
- Define your views (context, container, component, dynamic, deployment).
- Optionally add some styling (element styles, relationship styles, themes).
- And optionally create some documentation.
This approach provides a great way to build up and maintain your architecture diagrams as code, making it easier to track changes and updates.
Explain TraderX
So at this point, I needed to tell ChatGPT specifically what we were going to work with. ChatGPT won't access public GitHib repositories, even if you ask it to, so I next need to give it the actual DSL from the TraderX repo. Note, below I am only showing the model and relationships and excluding the views and styling for clarity, but ChatGPT is happy to be given that information too.
Great, I'm going to give you the C4 DSL for a trading system called TraderX
workspace "FINOS TraderX Sample Application" "An example distributed system in finance." {
model {
trader = person "Trader" "Managing Accounts and Creating Trades" "Trader"
tradingSystem = softwaresystem "Simple Trading System" "Allows employees to create accounts, and execute trades and view positions on these accounts." "Spring Boot Application" {
webFrontend = container "Web GUI" "Allows employees to manage accounts and book trades." "HTML and JavaScript and NodeJS"
peopleService = container "People Service" "Provides user details" ".NET Core"
accountService = container "Account Service" "Allows employees to manage accounts" "Java and Spring Boot"
positionService = container "Position Service" "View all trades and positions for an account" "Python and Flask"
refDataService = container "Reference Data Service" "Provides REST API to securities reference data" "NodeJS"
tradingService = container "Trading Services" "Allows employees create and update trades" "Java and Spring Boot"
messagebus = container "Trade Feed" "Message bus for streaming updates to trades and positions" "Topic-based Publish-subscribe engine" "SocketIO"
tradeProcessor = container "Trade Processor" "Process incoming trade requests, settle, and persist" "Java and Spring Boot"
database = container "Database" "Stores account, trade, and position state." "Relational database schema" "H2 Standalone"
}
userDirectory = softwaresystem "User Directory" "Golden source of User Data" "External"
webFrontend -> peopleService "Looks up people data based on typeahead from GUI" "REST/JSON/HTTP"
webFrontend -> refDataService "Looks up securities to assist with creating a trade ticket" "REST/JSON/HTTP"
peopleService -> userDirectory "Looks up people data" "LDAP"
webFrontend -> tradingService "Creatse new Trades and Cancel existing trades" "REST/JSON/HTTP"
tradingService -> messagebus "Publishes updates to trades and positions after persisting in the DB" "JSON/HTTP"
tradeProcessor -> messagebus "Processes incoming trade requests, persist, and publish updates" "SocketIO/JSON"
webFrontend -> accountService "Creates/Updates Accounts. Gets list of accounts" "REST/JSON/HTTP"
webFrontend -> positionService "Loads positions for account" "REST/JSON/HTTP"
webFrontend -> positionService "Loads trades for account" "REST/JSON/HTTP"
accountService -> database "CRUD operations around accounts." "SQL"
positionService -> database "Looks up default positions for a given account" "SQL"
positionService -> database "Looks up all trades for a given account" "SQL"
webFrontend -> messagebus "Subscribes to trade/position updates feed for currently viewed account" "WebSocket/JSON/WS"
tradeProcessor -> database "Looks up current positions when bootstraping state, persist trade state and position state" "SQL"
accountService -> peopleService "Validates People IDs when creating/modifying accounts" "REST/JSON/HTTP"
tradingService -> accountService "Validates accounts when creating trades" "REST/JSON/HTTP"
tradingService -> refDataService "Validates securities when creating trades" "REST/JSON/HTTP"
trader -> webFrontend "Manage Accounts"
trader -> webFrontend "Execute Trades"
trader -> webFrontend "View Trade Status / Positions"
}
Please summarise what you know about the application.
ChatGPT shows it has learned
So this is where things stsrt to get interesting, ChatGPT shows that it has understood key architectural facts about our system simply from understanding the C4 concepts and being able to interpret those in the context of the DSL.
Based on the Structurizr DSL provided, here is a summary of the TraderX trading system:
TraderX is a distributed system in finance that allows employees to manage accounts, execute trades, and view positions. The application is a Spring Boot Application.
TraderX consists of several services (containers) each responsible for a particular function and built with different technologies:
Web GUI: An interface for managing accounts and booking trades. Built with HTML, JavaScript, and NodeJS.
People Service: Provides user details, implemented with .NET Core.
Account Service: Allows employees to manage accounts. Implemented using Java and Spring Boot.
Position Service: Allows users to view all trades and positions for an account. Built with Python and Flask.
Reference Data Service: Provides securities reference data through a REST API, implemented with NodeJS.
Trading Services: Helps employees create and update trades. Implemented using Java and Spring Boot.
Trade Feed (message bus): A topic-based publish-subscribe engine used for streaming updates to trades and positions, implemented with SocketIO.
Trade Processor: Processes incoming trade requests, settle, and persist. Built with Java and Spring Boot.
Database: Stores account, trade, and position state. It's a relational database schema implemented with H2 Standalone.
There's also an External System named User Directory, which is considered the golden source of User Data. People Service communicates with this system to look up user data via LDAP.
The Trader interacts with the system primarily through the Web GUI. They can manage accounts, execute trades, and view trade status/positions.
Communication between services is generally done using REST/JSON/HTTP. The Trade Feed (message bus) and the Trade Processor also use SocketIO/JSON. The Web GUI subscribes to trade/position updates for currently viewed accounts via WebSocket/JSON/WS.
Database operations (CRUD operations around accounts, lookup default positions for a given account, lookup all trades for a given account, etc.) are performed using SQL.
In summary, TraderX is a microservices-based trading system, where each service is responsible for a specific functionality. The system leverages different technologies based on the appropriate use case, and ensures data consistency and communication between services via a message bus. It also uses a relational database to maintain and manage the state of accounts, trades, and positions.
Delving deeper
So, let's see if ChatGPT can answer questions about the relationships between the components in the system.
Please list the technologies used in each relationship between containers
Based on the Structurizr DSL you provided, here's a summary of the communication technologies used in each relationship between containers in the TraderX system:
Web GUI -> People Service: This relationship uses REST/JSON/HTTP for looking up people data based on typeahead from GUI.
Web GUI -> Reference Data Service: This relationship uses REST/JSON/HTTP for looking up securities to assist with creating a trade ticket.
People Service -> User Directory: This relationship uses LDAP for looking up people data.
Web GUI -> Trading Service: This relationship uses REST/JSON/HTTP for creating new Trades and cancelling existing ones.
Trading Service -> Trade Feed (message bus): This relationship uses JSON/HTTP for publishing updates to trades and positions after persisting in the DB.
Trade Processor -> Trade Feed (message bus): This relationship uses SocketIO/JSON for processing incoming trade requests, persisting, and publishing updates.
Web GUI -> Account Service: This relationship uses REST/JSON/HTTP for creating/updating Accounts and getting a list of accounts.
Web GUI -> Position Service: This relationship uses REST/JSON/HTTP for loading positions for account and also for loading trades for an account.
Account Service -> Database: This relationship uses SQL for CRUD operations around accounts.
Position Service -> Database: This relationship uses SQL for looking up default positions and all trades for a given account.
Web GUI -> Trade Feed (message bus): This relationship uses WebSocket/JSON/WS for subscribing to trade/position updates feed for currently viewed account.
Trade Processor -> Database: This relationship uses SQL for looking up current positions when bootstrapping state, persisting trade state, and position state.
Account Service -> People Service: This relationship uses REST/JSON/HTTP for validating People IDs when creating/modifying accounts.
Trading Service -> Account Service: This relationship uses REST/JSON/HTTP for validating accounts when creating trades.
Trading Service -> Reference Data Service: This relationship uses REST/JSON/HTTP for validating securities when creating trades.
This communication between the services is vital for the operation of the TraderX trading system, with different protocols and data formats used depending on the requirements of the specific interaction.
Can you aggregate how many relationships each technology supports
Based on the provided Structurizr DSL, here is the summary of how many relationships each technology supports:
- REST/JSON/HTTP: This is the most commonly used protocol in the system. It is used for 9 relationships:
- Web GUI -> People Service
- Web GUI -> Reference Data Service
- Web GUI -> Trading Service
- Web GUI -> Account Service (for both creating/updating accounts and getting list of accounts)
- Web GUI -> Position Service (for loading positions and trades for an account)
- Account Service -> People Service
- Trading Service -> Account Service
- Trading Service -> Reference Data Service
- SQL: Used in 3 relationships, mainly for CRUD operations and data lookups in the database:
- Account Service -> Database
- Position Service -> Database (for looking up positions and trades)
- Trade Processor -> Database
- JSON/HTTP: Used in one relationship for publishing updates to trades and positions:
- Trading Service -> Trade Feed (message bus)
- SocketIO/JSON: Used in one relationship for processing and publishing trade updates:
- Trade Processor -> Trade Feed (message bus)
- WebSocket/JSON/WS: Used in one relationship for real-time updates:
- Web GUI -> Trade Feed (message bus)
- LDAP: Used in one relationship for looking up user data:
- People Service -> User Directory
Each technology is employed based on its advantages and suitability to the needs of the specific interactions between services. For example, REST/JSON/HTTP is commonly used for synchronous interactions between services, while WebSocket/JSON/WS is used for real-time, bidirectional communication. SQL is used for interactions with the database, and LDAP for accessing user data from the User Directory system.
Here we see that ChatGPT has understood the relationships between the components and is able to answer questions about them. This is a great example of how we can use the information we capture as part of our architecture processes to make it easier for people to understand our systems.
However, we can also see that ChatGPT is not perfect, it has missed that JSON/HTTP is really the same as REST/JSON/HTTP.
Making corrections
JSON/HTTP is actually the same as REST/JSON/HTTP can you update your summary with the knowledge
Thank you for pointing that out. You're correct, both JSON/HTTP and REST/JSON/HTTP refer to using HTTP as the communication protocol and JSON as the data format. REST additionally implies a certain architectural style, but in practice, these terms are often used interchangeably.
Considering this, the updated summary would be:
- REST/JSON/HTTP (or JSON/HTTP): This protocol is used for 10 relationships:
- Web GUI -> People Service
- Web GUI -> Reference Data Service
- Web GUI -> Trading Service
- Trading Service -> Trade Feed (message bus)
- Web GUI -> Account Service (for both creating/updating accounts and getting list of accounts)
- Web GUI -> Position Service (for loading positions and trades for an account)
- Account Service -> People Service
- Trading Service -> Account Service
- Trading Service -> Reference Data Service
- SQL: Used in 3 relationships, mainly for CRUD operations and data lookups in the database:
- Account Service -> Database
- Position Service -> Database (for looking up positions and trades)
- Trade Processor -> Database
- SocketIO/JSON: Used in one relationship for processing and publishing trade updates:
- Trade Processor -> Trade Feed (message bus)
- WebSocket/JSON/WS: Used in one relationship for real-time updates:
- Web GUI -> Trade Feed (message bus)
- LDAP: Used in one relationship for looking up user data:
- People Service -> User Directory
So, the most commonly used protocol in the system is REST/JSON/HTTP (or JSON/HTTP) which is used in 10 relationships. The other protocols, SQL, SocketIO/JSON, WebSocket/JSON/WS, and LDAP, are used for specific use-cases.
This is a real world example of something I've come to see through a lot of my experiments with ChatGPT. Slight differences in the way we explain things can throw ChatGPT off or even result in it hallucinating information.
To really empower ChatGPT to do it's best work, having machine readable instructions such as APIs and schemas where exact relationships can be defined or where structure around information, such as using strong types, can be used to ensure ChatGPT has the best chance of understanding what we are asking it.
A Robo-Architect is born
We'll dig into this in more detail in future posts, for now, lets see if ChatGPT can take related, but distinct information and use it to make proposals about changes we may want to make to our architecture.
A frequent desire of a software architect is to drive consistency and ensure compliance with standards. Let's tell ChatGPT about a fictitious architecture policy that we want TraderX to be compliant with.
We have an architecture policy that we should favour using Google Protocol Buffers over JSON in REST calls. Could you recommend any changes we should make to the architecture and give me an updated C4 DSL reflecting those recommendations.
Absolutely. It's possible to replace JSON with Protocol Buffers (often abbreviated as Protobuf) in your REST calls. Protocol Buffers are language-agnostic, platform-neutral, extensible mechanism for serializing structured data developed by Google. This can bring benefits such as improved performance and strong data typing.
To make this shift in your architecture, you would need to make the necessary changes to your services that are currently using REST/JSON/HTTP. In your case, these are the following relationships:
- Web GUI -> People Service
- Web GUI -> Reference Data Service
- Web GUI -> Trading Service
- Trading Service -> Trade Feed (message bus)
- Web GUI -> Account Service (for both creating/updating accounts and getting list of accounts)
- Web GUI -> Position Service (for loading positions and trades for an account)
- Account Service -> People Service
- Trading Service -> Account Service
- Trading Service -> Reference Data Service
For each of these, you'll need to update your application logic to serialize and deserialize data using Protobuf instead of JSON. You will also need to update your API documentation to reflect this change.
Here's how your updated C4 model would look:
workspace "FINOS TraderX Sample Application" "An example distributed system in finance." {
model {
trader = person "Trader" "Managing Accounts and Creating Trades" "Trader"
tradingSystem = softwaresystem "Simple Trading System" "Allows employees to create accounts, and execute trades and view positions on these accounts." "Spring Boot Application" {
webFrontend = container "Web GUI" "Allows employees to manage accounts and book trades." "HTML and JavaScript and NodeJS"
peopleService = container "People Service" "Provides user details" ".NET Core"
accountService = container "Account Service" "Allows employees to manage accounts" "Java and Spring Boot"
positionService = container "Position Service" "View all trades and positions for an account" "Python and Flask"
refDataService = container "Reference Data Service" "Provides REST API to securities reference data" "NodeJS"
tradingService = container "Trading Services" "Allows employees create and update trades" "Java and Spring Boot"
messagebus = container "Trade Feed" "Message bus for streaming updates to trades and positions" "Topic-based Publish-subscribe engine" "SocketIO"
tradeProcessor = container "Trade Processor" "Process incoming trade requests, settle, and persist" "Java and Spring Boot"
database = container "Database" "Stores account, trade, and position state." "Relational database schema" "H2 Standalone"
}
userDirectory = softwaresystem "User Directory" "Golden source of User Data" "External"
webFrontend -> peopleService "Looks up people data based on typeahead from GUI" "REST/Protobuf/HTTP"
webFrontend -> refDataService "Looks up securities to assist with creating a trade ticket" "REST/Protobuf/HTTP"
peopleService -> userDirectory "Looks up people data" "LDAP"
webFrontend -> tradingService "Creates new Trades and Cancel existing trades" "REST/Protobuf/HTTP"
tradingService -> messagebus "Publishes updates to trades and positions after persisting in the DB" "REST/Protobuf/HTTP"
tradeProcessor -> messagebus "Processes incoming trade requests, persist, and publish updates" "SocketIO/JSON"
webFrontend -> accountService "Creates/Updates Accounts. Gets list of accounts" "REST/Protobuf/HTTP"
webFrontend -> positionService "Loads positions for account" "REST/Protobuf/HTTP"
webFrontend -> positionService "Loads trades for account" "REST/Protobuf/HTTP"
accountService -> database "CRUD operations around accounts." "SQL"
positionService -> database "Looks up default positions for a given account" "SQL"
positionService -> database "Looks up all trades for a given account" "SQL"
webFrontend -> messagebus "Subscribes to trade/position updates feed for currently viewed account" "WebSocket/JSON/WS"
tradeProcessor -> database "Looks up current positions when bootstraping state, persist trade state and position state" "SQL"
accountService -> peopleService "Validates People IDs when creating/modifying accounts" "REST/Protobuf/HTTP"
tradingService -> accountService "Validates accounts when creating trades" "REST/Protobuf/HTTP"
tradingService -> refDataService "Validates securities when creating trades" "REST/Protobuf/HTTP"
trader -> webFrontend "Manage Accounts"
trader -> webFrontend "Execute Trades"
trader -> webFrontend "View Trade Status / Positions"
}
Remember, replacing JSON with Protocol Buffers for your RESTful services will require changes in your implementation to serialize/deserialize data to/from Protocol Buffers format. It may also require changes on the client side, if external systems are consuming your services. It is recommended to fully assess these changes before moving forward.
So, whilst this is a simplistic example, not only hs ChatGPT been able to understand the policy in relation to the architecture and make recommendations in line with it, it has also been able to provide guidance that simply changing the REST calls to use Protocol Buffers is not enough, we also need to update the implementation to use Protocol Buffers.
Exciting possibilities
Thinking about this in the context of Architecture as Code this is really exciting as it already shows GenAI as have the potential to provide systematic architectural assurance, advice on necessary changes to comply with policy and advice on what else a software developer would need to consider when implementing changes to an existing architecture.
This is a topic we will be exploring in more detail in future posts. For now, thanks for reading.