What do you think about this API?

My intention is to create an API as generic and DRY as possible using Go. To achieve this, I have made some more less common decisions:

  1. To use AJAX call to avoid reloading the entire page (causing flickering) when updating the web page. Instead of using Go.
  2. To exclude hard coded queries in the API to reduce the endpoints (routes). As a bonus the queries can be modified and added in database without recompile the API when updating queries
  3. To use JSON to create and update data to get it more generic.
  4. To use the sqlx driver in order to further reduce code and avoid repeating.

enter image description here

My questions are:

  1. Can you see any security issues? (Except CORS)
  2. Anything you should done different?
  3. Any thoughts about the generic approach?

More detailed description is here: https://crud.go4webdev.org/api3rest

There doesn’t seem to be any authentication and/or authorization in the system, or at least I’m not seeing it. Is anyone allowed to do anything?

As for the code itself, I’d recommend trying for a more layered approach. Calls go all over the place, which makes it harder to read. You’ll find that as the system evolves it will become messier still. Better to think out some modules beforehand (like query, handler, etc) and implement them from the start to get good separation of concerns.

As with all CRUD systems it’ll probably take you a long way into solving generic problems, but as soon as you step away from those into more complicated problems you enter a world of hurt, but that goes for all CRUD systems, not just yours.

1 Like

Thank your for your feedback! Yes. As a draft it is wide open.

Can you elaborate what you mean with “layers”? As shown here I have 6 layers (to me).

Any example?

What I mean by layers is a grouping of code that has one responsibility. For example, your create, read, update and delete would be a single layer, concerned with database operations. The HTTP server is another layer, concerned with HTTP only.

Below the create, read, update and delete there could be a database abstraction layer, so the the create, read, update and delete can express themselves in more generic terms and then the database layer can figure out what that mean. This makes it easier to swap out MySQL for Postgres or maybe something like Redis.

In essence I’m talking about Layered Architecture.

Suppose you want to create a service for trading commodities. When I want to buy something, it’s not as easy as just creation a “buy” record. You also need to figure out in the system at what price I’m buying.

Or when I do something that requires two scopes (as you call them) to be created at the same time. For example a user and the company they work for. What if creating the company succeeds, but creating the user fails? You can’t redo the whole thing, because the company already exists and you can’t recreate it. And if the user decides to give up and leave you’re left with a company in the database that isn’t linked to anything.

I’m not saying CRUD is never useful, but you do need to realise when it is and when it isn’t.

1 Like

If I understand you correct, my draft consists of 3.5 layers :slight_smile: Presentation Layer (Web stuff), Persistence Layer (API) and Database layer. The partial business layer may be the Go engine that serves the Web site. I have considered a separate business layer, but I have to (as mentioned in the article) deal with speed as well. For now, I put the business rules where ever they make sense. In my reality there is no 100 percent isolation of the business layer. You may put some intelligence in your SQL function as well. When this comes to production, I will consider all 4 layers…

Yes, this is the main reason I do not follow the standard for this draft. Today I have ancient slow code that fetches each table separate and make some calculation at the front. My recent code just fire a query and let Postgresql do this calculation in one step. Way faster, but breaks the “business layer” principle. Do I understand you correct?

The general approach sounds fine. However, I would also recommend distributed services across micro services instead of one gigantic monolith. This can be achieved by using a mono repo design pattern. Also for deployment I would highly recommend aws lambda and using server less framework. Serverless allows you to focus more on the application than infrastructure. I would also like to share a link of a go project that I believe could provide some insight into useful architectural patterns.

These are some key attributes to point out.

  • Organizing go lambda api in a mono repo using bazel.
  • Integration of go with serverless framework
  • Integration with aws cognito for authentication
  • Extensible entity manager with pluggable storage mechanism currently supporting aws s3, Cassandra, and elastic search.
  • Using go templates to build dynamic queries eliminating the need to hard code queries or use a bulky orm.

Unfortunately over the last year more focus has been on the front-end focus than this. In particularly achieving zero trust communication with all the after mentioned aws services directly in the browser using signed http requests. Actually eliminating the need completely for a middle layer like this api. So the goal has been to migrate completely off this api and instead achieve zero trust communication directly with cloud services in the browser. I think when devs who are using cloud services discover that those services can be securely spoken to in the browser much of the need for these apis will go away and be replaced with simpler applications that communicate directly with zero trust cloud.

Thank you for the feedback and tips. Many terms are new to me, so I have to Google it :slight_smile:

The American CLOUD Act means that you theoritically can be sued if you use AWS or similar. Any other suggestion?

In that case it might be better to focus on front-end application than creating a go api. I have moved away from relational databases for lower-latency, highly available, scalable cloud services but you could use post graph ql directly to a database server. That would allow you to build the queries or graph statement directly in the browser and eliminate the need to have a go api in the first place or would greatly simplify the the go api since you would only really need one single end-point to process a graph query.

I have not used this myself but aws amplify actually provides a model for such development.

This might get you off to the races with a cloud database that can process graph ql using postres. No longer do you need a go api. Instead you can communicate directly with the server in the browser using graph ql.

Yes, I have considered using ORM or smilar in order to simplify. Every ORM I have seen is like JQuery. Another language upon another language. Many ORM’s covers 60 percent of real queries very easy. But the remaining 40 percent will be bigger or much bigger challenge. For an example I do not know any ORM that can manage CTE statement (WITH etc), which I use frequently.

And without an API, how can the partners reach the database?

So I decided to use Vanilla in order to reduce the number of languages I have to learn and maintain :slight_smile:

Vanilla js might seem like a good idea and it can be for a learning experience. However, long term projects as complexity increases simplicity will be out-weighed by the need for boiler plate code and patterns for scalable software easily to maintain and onboard others.

Exactly. Do what works for your business, don’t horse shoe it into whatever your technical solution happens to be.

Yes you understand, but this doesn’t break the business layer. Nobody ever said the business layer : persistence layer needs to be 1:1, or that the business layer isn’t allowed to ask intelligent queries.
The main reason the business layer is there is to decouple what is asked from storage1 from how that is asked. That way, if you swap out Postgres for MySQL for example, you only need to change the persistence layer; the business and presentation layer can remain as they are.

Now, people don’t swap out databases often, but also consider outdated APIs for talking to databases. When you need to upgrade those you only have to change the persistence layer, which is a lot easier than having to change things all over the place.

1 Using a generic term here on purpose. Mostly this would be a database, but not necessarily. It could also be a file on disk, or just an array in memory (for automated testing).

As Sam Newman, the inventor of microservices would tell you, they are not a way to scale software, but a way to scale organizations. So far we have evidence of a single person working on this. I fail to see how microservices would help here.

1 Like

So if I use GraphQL there will only be 2 layers? Frontend and Database. Do I understand this correct?

In my example the REST API is a microservice sort of. And I can think of an isolated microservice for database creation and updating. As this microservice is called very seldom, it may simplify to maintain this process without interfering with the daily work. And Go is more or less built for microservices. So @windbeneathmywings may have a point here. But I agree not for a scaling purpose alone.

I would still use 3, one intermediate to translate what HTTP says to what is requested from any backends.

I would seriously consider setting up a graph QL sever that the browser can directly communicate with using zero-trust authentication. With this model the stack is simplified and entire traditional middle layer/man is eliminated. The one thing I’m not familiar with though is whether fine grain access control can easily be introduced into this structure without a middle server layer.

This topic was automatically closed 91 days after the last reply. New replies are no longer allowed.