Linux Daemon with Go: Build a Webpage Database Crawler from Coursera

More specifically, in this 1.5-hour long project-based course, you will develop a Linux daemon, communicate with web servers using HTTP, as well as extract key data from textual raw data.

To achieve this, you will create a web article database crawler application by working through a scenario where you are an intermediate-level Go developer for a tech company.

In order to be successful in this project, you will need to know the fundamentals of Go including slices, functions, and error handling. MySQL connection and row update with Go ("Relational Database with Go: Build an Article Search Engine" highly recommended).

What's inside

Syllabus

Project Overview

Do you know that over 75% of projects in the Cloud Native Computing Foundation, including Docker and Kubernetes, are written in Go? This Guided Project was created to help learners create database applications that can run as background processes or daemons using Go. More specifically, in this 1.5-hour long project-based course, you will learn how develop Linux daemon, communicating with web servers using HTTP, as well as extracting key data from textual raw data. To achieve this, you will create a web article database crawler application by working through a scenario where you are an intermediate-level Go developer for a tech company. In order to be successful in this project, you will need to know the fundamentals of Go including slices, functions, and error handling. MySQL connection and row update with Go ("Relational Database with Go: Build an Article Search Engine" highly recommended).

Good to know

Know what's good

, what to watch for

, and possible dealbreakers

Uses Go, which is a popular language for cloud-native projects like Docker and Kubernetes, making it highly relevant for developers in that space

Develops a Linux daemon, which is useful for creating background processes and services, a common task in systems programming and server-side development

Teaches HTTP communication, which is essential for building web applications and interacting with APIs, a core skill for web developers

Recommends prior experience with MySQL connection and row updates with Go, suggesting that learners should already have a foundation in database interactions

Requires familiarity with Go fundamentals like slices, functions, and error handling, indicating that this project is designed for intermediate-level learners

Focuses on extracting key data from textual raw data, which is a valuable skill for data processing and analysis in various applications

Activities

Be better prepared before your course. Deepen your understanding during and after it. Supplement your coursework and achieve mastery of the topics covered in Linux Daemon with Go: Build a Webpage Database Crawler with these activities:

Review Go Fundamentals

Show steps

Reinforce your understanding of Go fundamentals, including slices, functions, and error handling, to ensure a solid foundation for the course.

Browse courses on Go Programming

Show steps

Review Go syntax and data types.
Practice writing functions and using slices.
Study error handling techniques in Go.

Review Go Fundamentals

Show steps

Reinforce your understanding of Go fundamentals, including slices, functions, and error handling, to ensure a solid foundation for the course.

Show steps

Review Go documentation on slices, functions, and error handling.
Complete practice exercises on Go slices, functions, and error handling.

Review 'The Go Programming Language'

Show steps

Deepen your understanding of Go by studying a comprehensive guide to the language.

View Go Programming Language, The (Addison-Wesley... on Amazon

Show steps

Read the chapters relevant to web servers, HTTP requests, and data extraction.
Experiment with the code examples provided in the book.

Five other activities

Expand to see all activities and additional details

Show all eight activities

Follow HTTP Request Tutorials in Go

Show steps

Refine your skills in making HTTP requests with Go by following online tutorials.

Browse courses on HTTP Requests

Show steps

Search for tutorials on making HTTP requests in Go.
Implement the examples provided in the tutorials.
Modify the examples to suit your specific needs.

Review 'Go in Action'

Show steps

Gain practical insights into building real-world applications with Go by studying a hands-on guide.

View Go in Action on Amazon

Show steps

Read the chapters relevant to concurrency and networking.
Implement the examples provided in the book.

Build a Simple Web Scraper

Show steps

Solidify your understanding of web scraping by building a simple web scraper that extracts data from a website.

Browse courses on HTTP

Show steps

Parse the HTML content to extract the desired data.
Choose a website to scrape.
Write Go code to fetch the website's HTML content.
Store the extracted data in a file or database.

Document Your Web Crawler Project

Show steps

Improve your understanding and retention by documenting the web crawler project you built in the course.

Browse courses on Documentation

Show steps

Write a detailed explanation of the project's architecture.
Document the purpose and functionality of each code module.
Create a user guide for the web crawler.

Contribute to a Go Project

Show steps

Deepen your understanding of Go and software development by contributing to an open-source Go project.

Browse courses on Open Source

Show steps

Find an open-source Go project that interests you.
Read the project's documentation and contribution guidelines.
Identify a bug or feature to work on.
Submit a pull request with your changes.

Career center

Learners who complete Linux Daemon with Go: Build a Webpage Database Crawler will develop knowledge and skills that may be useful to these careers:

Backend Engineer

A backend engineer focuses on the server-side logic and databases that power applications. This role often involves creating and maintaining robust systems that handle data processing and storage. This course helps one build a foundation in creating database applications using Go, a language that is prevalent in cloud-native technologies and often used in backend development. The course’s focus on building a Linux daemon, communicating with web servers using HTTP, and extracting data from raw text all translate directly into skills a backend engineer needs to be successful. Knowledge of error handling as well as experience with MySQL is also crucial.

See salaries and explore the career path for Backend Engineer

Software Developer

A software developer creates applications and systems that run on computers. This can involve working on web applications, mobile apps, or internal tools. This course helps individuals develop the ability to create programs, specifically a database crawler, that run as background processes or daemons using Go. The skills gained in working with web servers using HTTP, extracting data from text, and practical work with Go are all integral to various development projects. A software developer should be comfortable with these topics, and the course's project-based approach provides useful experience.

See salaries and explore the career path for Software Developer

Application Developer

An application developer is responsible for designing, coding, testing and deploying applications for various platforms such as desktop or mobile. The role requires a strong understanding of programming fundamentals and the ability to build robust software. This course helps an application developer by providing direct experience in creating a web article database crawler using Go. The skills gained in building Linux daemons, communicating with web servers using HTTP, and extracting data from raw text enable programmers to develop applications that involve a diverse range of technologies, and helps to build a strong foundation in Go.

See salaries and explore the career path for Application Developer

System Administrator

A system administrator manages and maintains the computer systems and servers that an organization relies on. This can include ensuring the smooth operation of services and applications. This course helps build a background in creating and managing Linux daemons. These background processes form a critical component of many distributed systems and require skilled management. Those who want to learn to manage server-side processes should find the practical experience useful.

See salaries and explore the career path for System Administrator

Cloud Engineer

A cloud engineer designs, implements and manages cloud computing solutions. This role involves understanding various cloud platforms and services. This course may be helpful for cloud engineers, particularly those working with Go, a language that is used by over 75% of projects in the Cloud Native Computing Foundation. The focus on creating background processes and communication with web servers could be directly useful in managing various cloud application components. This course helps a cloud engineer understand how to build and deploy applications that are designed for the cloud.

See salaries and explore the career path for Cloud Engineer

Data Engineer

A data engineer builds and maintains the infrastructure required for data analysis and machine learning. This role includes building data pipelines and dealing with large datasets. This course may be useful for data engineers, especially those who work with data ingestion and processing. The course provides key skills in building a web article database crawler that extracts key data from text and runs as a background process. A data engineer may find the experience in web server communication and data extraction to be a useful exercise.

See salaries and explore the career path for Data Engineer

DevOps Engineer

A DevOps engineer works to streamline software development and deployment processes. They often combine software development and operations to ensure efficient and reliable deployments. This course may be useful for DevOps engineers because it dives into creating and managing background processes, a crucial component of many applications. The skill of building a Linux daemon that can communicate with web servers using HTTP, and extract data from raw text, directly helps one understand systems that a DevOps engineer often needs to implement or maintain. The project-based nature of the course can help one get familiar with practical aspects of deployment and automation.

See salaries and explore the career path for DevOps Engineer

Site Reliability Engineer

A site reliability engineer focuses on the performance, stability, and availability of applications and services. They work to ensure systems are reliable and can handle traffic and maintain a good user experience. This course may help build an understanding of how the services that an SRE is managing are built. The practical skills obtained in this course are useful for those who need to learn how to create background processes or services, as well as how to communicate with web servers and extract data. The course builds an understanding of creating and managing Linux daemons which form the basis of many distributed systems.

See salaries and explore the career path for Site Reliability Engineer

Web Developer

A web developer designs and builds websites and web applications. This role includes working on both the front-end and back-end aspects of a website. A web developer may find this course helpful, as it gives them an understanding of server-side processes. This course in particular focuses on the back-end, where a web developer can learn how to create database applications that can run as background processes or daemons using Go, how to communicate with web servers using HTTP, and extract key data. All these skills can help improve a web application.

See salaries and explore the career path for Web Developer

Database Administrator

A database administrator is responsible for the performance, integrity, and security of a database. They manage and maintain the databases that an organization depends on. This course may help those who work with databases, specifically in the context of relational databases like MySQL. The course project involves creating a web article database crawler, which requires working with MySQL connections and row updates. Those with a focus on databases can benefit from the course's focus on database application development using Go.

See salaries and explore the career path for Database Administrator

Data Analyst

A data analyst is responsible for collecting, cleaning, and interpreting data. They use their analytical skills to generate insights and recommendations. A data analyst may find this course helpful in learning how data is gathered and processed automatically. The course's focus on creating a web article database crawler provides insight into how data can be extracted from raw text, and stored in a database. It is a good exercise in the tools to build and manage data pipelines to make data available for analysis.

See salaries and explore the career path for Data Analyst

Technical Project Manager

A technical project manager is a professional who leads a team through a technical project. They are responsible for planning, executing, and coordinating these projects. This course may help a project manager gain a deeper understanding of the development process. The course provides key insights into the steps involved in database application development, including working with Linux daemons, HTTP communications, and data extraction. This could help a manager in managing more effectively and asking the correct questions.

See salaries and explore the career path for Technical Project Manager

Technical Support Specialist

A technical support specialist provides customer support related to technical product and applications. They troubleshoot technical issues and help customers to work with technology. This role may have some overlap with the course, which can help a support specialist gain an understanding of how software works. Technical support will be able to better assist customers by understanding the background processes, HTTP interaction, and data extraction that may occur.

See salaries and explore the career path for Technical Support Specialist

Information Security Analyst

An information security analyst plans, implements, upgrades, and monitors the security measures for the protection of computer networks and information. They identify threats and risks to computer systems and work to mitigate them. This course may help build some background in how systems are built. Understanding how background processes are created, how information is transmitted through HTTP and how databases work will help an information security analyst. A deeper understanding of the software helps one assess risk.

See salaries and explore the career path for Information Security Analyst

Network Engineer

A network engineer designs, sets up, and maintains computer networks. They ensure the smooth functioning of network infrastructure. This course may be useful to a network engineer, especially those who deal with web servers. The course emphasizes communication with web servers using HTTP. Furthermore, understanding how data is extracted and processed will help a network engineer better understand data flow.

See salaries and explore the career path for Network Engineer

Reading list

We've selected two books that we think will supplement your learning. Use these to develop background knowledge, enrich your coursework, and gain a deeper understanding of the topics covered in Linux Daemon with Go: Build a Webpage Database Crawler.

The Go Programming Language

Save

Comprehensive guide to the Go programming language. It covers the language's syntax, semantics, and standard library in detail. It is commonly used as a textbook at academic institutions and by industry professionals. Reading this book will provide a deeper understanding of Go and its capabilities, enhancing your ability to build robust daemons and web crawlers.

Go Programming Language, The (Addison-Wesley...

Kindle Edition

Go in Action

Save

Provides a practical, hands-on approach to learning Go. It focuses on building real-world applications and solving common problems. It valuable resource for intermediate-level Go developers. Reading this book will give you practical insights into building robust and scalable applications with Go, complementing the knowledge gained in the course.

Linux Daemon with Go

Build a Webpage Database Crawler

Here's a deal for you

What's inside

Syllabus

Good to know

Save this course

Activities

Career center

Reading list

Share

Similar courses