Mirrors/cobalt
1
0
Fork 0
mirror of https://github.com/wukko/cobalt.git synced 2024-11-15 20:59:59 +00:00
Code Issues Projects Releases Packages Wiki Activity
cobalt/src/front/robots.txt

9 lines
148 B
Text
Raw Normal View History

moved to new repo
2022-07-08 19:17:56 +01:00
User-Agent: *
robots: added more paths to ignore
2023-12-26 12:28:47 +00:00
Disallow: /emoji/
fix robots.txt currently the robots.txt file is useless because it's interpreted as one path "/icons/ /fonts/ *.js *.css" (an example path that would be accepted -- and therefore disallowed for robots) by this regex would be `https://cobalt.tools/icons/ /fonts/ bla.js .css`, which is obviously nonsense & useless)
2023-11-10 17:27:18 +00:00
Disallow: /fonts/
robots: added more paths to ignore
2023-12-26 12:28:47 +00:00
Disallow: /icons/
Disallow: /sponsors/
Disallow: /updateBanners/
fix robots.txt currently the robots.txt file is useless because it's interpreted as one path "/icons/ /fonts/ *.js *.css" (an example path that would be accepted -- and therefore disallowed for robots) by this regex would be `https://cobalt.tools/icons/ /fonts/ bla.js .css`, which is obviously nonsense & useless)
2023-11-10 17:27:18 +00:00
Disallow: /*.js
Disallow: /*.css
Reference in a new issue Copy permalink