seoR (version 0.1.0)

isIndexable: Function to check if a URL is indexable

Description

This function checks if a given URL is indexable. Therfore it checks different rules: Status-Code: Does the URl gives a correct Status-Code (200) Canonical-Link: Is the canonical Link Element implementet, so the URL can be indexed? Meta-Robots: Is the Meta-Robots blocking the ULR? robots.txt: Is the robots.txt blocking the URL from crawling?

Usage

isIndexable(url, bot = "googlebot", errorType = TRUE)

Arguments

url

The url you want to check the indexability for

bot

The bot you want to check the indexability with. Default is googlebot

errorType

Logical. Retrieves the Error, why the URL can not be indexed. Attention not all rules at once are displayed. isIndexable()

Examples

# NOT RUN {
isIndexable("https://www.r-project.org/", bot = "googlebot", errorType = TRUE)
# }