A sitemap is an important part of any website. It stores all the URLs available on your website, usually in an HTML format or more commonly in an XML format. It allows your website to be accessed by web crawlers and retrieve necessary information using web scraping. Sitemaps can also boost your website’s SEO ranking since they make it easier for search engines to navigate through your website. They can be categorized into static and dynamic sitemaps and can be generated for both static and dynamic websites.
Sitemaps For Static Websites
Static websites contain hardcoded URLs and a fixed number of web pages. A sitemap for a static website can be created easily using an online sitemap generator tool such as XML-Sitemaps (http://www.xml-sitemaps.com).
To use a sitemap generator, you need to enter your website’s URL to automatically generate a sitemap. Here, we are creating a static sitemap that can only be altered manually by a developer. This method works best for websites with a smaller yet fixed number of web pages. An example of a static sitemap would be as follows
|
<?xml version=“1.0” encoding=“UTF-8”?>
<urlset xmlns=“http://www.sitemaps.org/schemas/sitemap/0.9”>
<url>
<loc>https://www.example.com/</loc>
<lastmod>2024-01-12T00:00:00+00:00</lastmod>
<changefreq>weekly</changefreq>
<priority>1.0</priority>
</url>
<url>
<loc>https://www.example.com/about-us</loc>
<lastmod>2024-01-12T00:00:00+00:00</lastmod>
<changefreq>monthly</changefreq>
<priority>0.8</priority>
</url>
<url>
<loc>https://www.example.com/contact</loc>
<lastmod>2024-01-12T00:00:00+00:00</lastmod>
<changefreq>monthly</changefreq>
<priority>0.7</priority>
</url>
</urlset>
|
We observe that this sitemap contains the URLs of the ‘home’ page, the ‘about’ page, and the ‘contact’ page.
Seemingly static websites created by WordPress make use of plugins etc to generate sitemaps automatically.
Sitemaps for Dynamic Websites
The code showed here is an actual example from NSE INDIA holiday API Portal.
Generating a sitemap for a dynamic website is relatively more complex since dynamic websites contain a variable number of web pages and support CRUD operations. Moreover, if a website contains authentication, web crawlers won’t be able to access the web pages visible to users after logging in. In this case, a static sitemap generator will not produce accurate results. Even if a sitemap is generated this way, developers would have to constantly track and update the sitemap based on changes occurring in the website.
Nevertheless, we can generate a dynamic sitemap for a website in Laravel. For this, we require three things – a controller to define the logic for creating the sitemap, a route to define the route of the sitemap, and a kernel function to automate the process of updating the sitemap periodically.
Creating The Controller
To create a controller we use the following command in Artisan.
|
php artisan make:controller SitemapController
|
A controller is created in the ‘app’ directory under the ‘Controllers’ folder. Now, we need to define the command name and provide a description of the command.
|
protected $signature = ‘sitemap:dynamic’;
protected $description = ‘Generate sitemap for the dynamic website’;
|
If you have any specific links that you would like to retain in the sitemap, you can declare them too. For that, you need to just define the base URL and the URLs of the websites that you would like to keep. Here, the base URL is typically denoted by ‘/’ but you can also use ‘app.url’ found in the .env file of your Laravel project.
|
$baseUrl = config(‘app.url’);
$sitemap = ‘<?xml version=”1.0″ encoding=”UTF-8″?>’ . PHP_EOL;
$sitemap .= ‘<urlset xmlns=”http://www.sitemaps.org/schemas/sitemap/0.9″>’ . PHP_EOL;
$fixedUrls = [ // define specific URLs
$baseUrl,
$baseUrl . ‘/login’,
$baseUrl . ‘/password/reset’,
];
|
We can use this piece of code to loop through the fixed routes and generate the XML for the static links in the sitemap.
|
foreach ($fixedUrls as $url)
{
$lastmod = Carbon::now()->toIso8601String();
$sitemap .= “\t<url>” . PHP_EOL; // <url> opening tag
$sitemap .= “\t\t<loc>$url</loc>” . PHP_EOL; // URL of the webpage
$sitemap .= “\t\t<lastmod>$lastmod</lastmod>” . PHP_EOL; // day last modified
$sitemap .= “\t\t<priority>1.00</priority>” . PHP_EOL; // priority of URL
$sitemap .= “\t</url>” . PHP_EOL; // <url> closing tag
}
|
Now if we have a few URLs containing dynamic parameters, we can extract them from our database and automate the process of generating URLs for each parameter, similar to the example shown above.
|
$entityIds = Entity::pluck(‘id’); // extract all ‘id’s
foreach ($entityIds as $entityId) // loop through ‘id’s
{
$url = $baseUrl . “/entity/$entityId/holiday/index”;
$lastmod = Carbon::now()->toIso8601String();
$sitemap .= “\t<url>” . PHP_EOL;
$sitemap .= “\t\t<loc>$url</loc>” . PHP_EOL;
$sitemap .= “\t\t<lastmod>$lastmod</lastmod>” . PHP_EOL;
$sitemap .= “\t\t<priority>0.80</priority>” . PHP_EOL;
$sitemap .= “\t</url>” . PHP_EOL;
}
$sitemap .= ‘</urlset>’ . PHP_EOL; // close the <urlset> tag
|
Finally, we can save the sitemap generated and conclude the process.
|
$sitemapPath = public_path(‘sitemap.xml’);
File::put($sitemapPath, $sitemap);
$this->info(‘Sitemap generated successfully.’);
$this->info(“Sitemap file saved at: $sitemapPath”);
$this->info(‘GenerateSitemap command completed.’);
|
So we should have the complete code as shown below.
|
<?php
namespace App\Console\Commands;
use Illuminate\Console\Command;
use Illuminate\Support\Facades\File;
use Carbon\Carbon;
use App\Models\Entity;
class GenerateSitemap extends Command
{
protected $signature = ‘sitemap:dynamic’;
protected $description = ‘Generate sitemap for the dynamic website’;
public function handle()
{
$this->info(‘GenerateSitemap command is running…’);
$baseUrl = config(‘app.url’);
$sitemap = ‘<?xml version=”1.0″ encoding=”UTF-8″?>’ . PHP_EOL;
$sitemap .= ‘<urlset xmlns=”http://www.sitemaps.org/schemas/sitemap/0.9″>’ . PHP_EOL;
$fixedUrls = [
$baseUrl,
$baseUrl . ‘/login’,
$baseUrl . ‘/password/reset’,
];
foreach ($fixedUrls as $url) {
$lastmod = Carbon::now()->toIso8601String();
$sitemap .= “\t<url>” . PHP_EOL;
$sitemap .= “\t\t<loc>$url</loc>” . PHP_EOL;
$sitemap .= “\t\t<lastmod>$lastmod</lastmod>” . PHP_EOL;
$sitemap .= “\t\t<priority>1.00</priority>” . PHP_EOL;
$sitemap .= “\t</url>” . PHP_EOL;
}
$entityIds = Entity::pluck(‘id’);
foreach ($entityIds as $entityId)
{
$url = $baseUrl . “/entity/$entityId/holiday/index”;
$lastmod = Carbon::now()->toIso8601String();
$sitemap .= “\t<url>” . PHP_EOL;
$sitemap .= “\t\t<loc>$url</loc>” . PHP_EOL;
$sitemap .= “\t\t<lastmod>$lastmod</lastmod>” . PHP_EOL;
$sitemap .= “\t\t<priority>0.80</priority>” . PHP_EOL;
$sitemap .= “\t</url>” . PHP_EOL;
}
$sitemap .= ‘</urlset>’ . PHP_EOL;
$sitemapPath = public_path(‘sitemap.xml’);
File::put($sitemapPath, $sitemap);
$this->info(‘Sitemap generated successfully.’);
$this->info(“Sitemap file saved at: $sitemapPath”);
$this->info(‘GenerateSitemap command completed.’);
}
}
|
Defining The Route
We can define the route of the sitemap in the ‘web.php’ file of our project.
|
Route::get(‘/sitemap.xml’, function()
{
return \Illuminate\Support\Facades\Redirect::to(‘sitemap.xml’);
});
|
Defining The Schedule Method In The Kernel
The script can be triggered automatically to run at a particular time.
|
use App\Console\Commands\GenerateSitemap; // define the controller path
|
|
protected function schedule(Schedule $schedule)
{
// Run the command daily at 5 PM
$schedule->command(GenerateSitemap::class)->dailyAt(’17:00′);
}
|
References:
- https://www.uplers.com/blog/all-about-of-sitemap/
- https://www.linkedin.com/pulse/understanding-difference-between-dynamic-static-sitemaps-arslan-elahi
