Sign in

Turning log data into real time insights. Real-time log aggregation and analysis to any S3 compatible object-store| Eliminate Cost Per GB |
Shift from API Monitoring to API Observability with LOGIQ

APIs — by now, we’re all familiar with the term. Every service or software we use or build today either uses or is an API. If APIs are a central pillar in your building and delivery of software and services, you’ll know that the success of your software or services depends on the integrity, availability, and performance of your APIs. Traditional API monitoring sure does help you stay on top of uptime, security, and performance. Still, it is limited to being a black box form of monitoring — you’re only testing API behavior that you’d only see externally. …

Tutorial: Convert your log data to a time-series event visualization

Log data contains tons of information — the more complex the system producing the logs, the more information the logs have. Imagine going through endless lines of log data to find an event that’s of interest to you. Not only is it going to sap away a lot of your time, but it’s also going to exhaust you mentally. Now imagine if you could instead visualize the occurrences of any particular event within your application or infrastructure as a time series just by plugging in your log stream.

LOGIQ lets you do just that. With LOGIQ insights, you can generate…

Simplifying SaaS, PaaS, and IaaS

Cloud computing is everywhere. An ever-increasing number of businesses have turned to the cloud for their storage, compute, analytics, content management and business operations demands. With its ever-increasing adoption, cloud computing is quickly turning into the lifeline for infrastructure, platform, and software setups for organizations of all sizes. The compelling and affordable pricing of cloud computing technology has made it a no-brainer for companies to adopt and build their operations and offerings around the cloud.

The cloud is pretty comprehensive — through the cloud, you get access to a wide variety of online services through various delivery models. You’ll often…

Compute & Storage Should Be Decoupled for Log Management at Scale

Most log management solutions store log data in a database and enable search by storing an index of the data. As the database grows in size, so does the index management cost. On a small scale, this isn’t problematic. But when dealing with large-scale deployments, organizations end up using lots of compute, storage and human resources just to manage their indexes, in addition to data itself. When companies are handling terabytes of data every day, the database-backed log management system becomes untenable.

Another common issue is that most log solutions don’t store just one set of data. Many DIY log…

If you’ve been looking for an inexpensive way to run your own observability stack while maintaining complete control over your data and its security, look no further. The LOGIQ PaaS Community Edition is officially live!

With the LOGIQ PaaS Community Edition, you can:

  • Self-host your observability stack on a cloud provider of your choice — public or private
  • Ingest up to 50GB of log data per day with unlimited data retention
  • Store your log data on any S3-compatible cloud provider via the built-in Minio S3 service
  • Ingest logs from Syslog, RSyslog, Logstash, Fluent, AWS Firelens, JSON, and plenty more
  • Run up to 4 ingest worker processes

You’ll also get access to all of the LOGIQ Enterprise Edition’s features along with Community Support, free forever.

There’s no one-size-fits-all approach regarding application monitoring, especially for companies using applications in various cloud environments. Companies are rapidly investing in microservices, mobile apps, data science programs, data ops, etc. Subsequently, they’re also integrating monitoring tools to improve domain-centric monitoring abilities.

AIOps tools help streamline the use of monitoring applications. It allows companies that need high application services to efficiently manage the complexities of IT workflows and monitoring tools. AIOps extends machine learning and automation abilities to IT operations. …

Logging and log management are critical aspects of modern IT management, monitoring, and security. Regardless of whether your IT systems follow monolithic or microservices architecture, they are complex due to the number of moving parts and the challenges they unwittingly pose around management, deployment, and scaling. Log management systems provide DevOps teams and SREs a third eye that helps monitor and improve performance, investigate errors, visualize events as they occur, and help them look ahead.

Log collectors, or aggregators, are critical aspects of the log management infrastructure. They help collect logs from various systems and parse and groom them for…

AWS CloudWatch is an observability and monitoring service that provides you with actionable insights to monitor your applications, stay on top of performance changes, and optimize resource utilization while providing a centralized view of operational health. AWS CloudWatch collects operational data of your AWS resources, applications, and services running on AWS and on-prem servers in the form of logs, metrics, and events. CloudWatch then uses this data to help detect and troubleshoot issues and errors in your environments, visualize logs and metrics, set up and take automated actions, and uncover insights that help keep your applications and deployments running smoothly.

We’re huge fans of Helm Charts and the simplicity they bring to complex application deployments on Kubernetes and MicroK8s. We showed you how you could use Helm Charts to deploy LOGIQ on MicroK8s in a previous post. As a follow-up to that article, we’d like to show you how Helm Charts are equally helpful in deploying complex applications on other certified Kubernetes distributions such as K3s. In this article, you’ll learn how we use Helm Charts to deploy the LOGIQ observability platform on K3s. If you’ve never used K3s or Helm Charts before, you can use this article to get acquainted with both.

What is K3s?

K3s is a highly available, lightweight, and fully compliant Kubernetes distribution. K3s is packaged as a single binary, thereby reducing dependencies and minimizing installation, run, and auto-update steps that you’d typically have to take while managing a production Kubernetes cluster. As K3s is lightweight, you can run a cluster on machines with as little as 512 MB RAM and upwards.

K3s comes bundled with a local storage provider, a service load balancer, a Helm controller, and the Traefik ingress controller. …

The growth in information architecture has urged many IT technologies to adopt cloud services and grow over time. Microservices have been the frontrunner in this regard and have grown exponentially in their popularity for designing diverse applications to be independently deployable services.

Trivia: In a survey by O’Reilly, over 50% of respondents said that more than 50% of new development in their organization utilizes microservices.

Using isolated modules, microservices in the Cloud stray away from using monolithic systems, where an entire application could fail due to a single error in a module. …

Amit Ashwini

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store