Zablokowanie stron robotom Google

[Amfidiusz]

Hej,

Zauważyłem, że Google zaczęło mi indeksować linki typu:

http://****/members/page__sort_key__joined__sort_order__asc__max_results__20__quickjump__Z__name_box__begins__name__Z__quickjump__P

Dodałem więc poniższy wpis w robots.txt:

Disallow: /*page__sort_key*

Jednak nie przyniósł on żadnego efektu. W jaki sposób mogę się pozbyć takich duplikatów?

[Macsch15]

http://www.forum.tweaks.pl/forum-IPB-usuwanie-duplikatow-stron-w-wynikach-Google-t44419.html

Google pewnie musi zaktualizować wyniki.

[Amfidiusz]

Czekam już od miesiąca i nadal nic, więc postanowiłem zapytać, czy czegoś czasami nie robię nie tak

[Macsch15]

Może zaktualizuj sitemap'e?

[mlodszy]

Tu masz duże robots.txt z community brane, z małymi zmianami moimi:

User-agent: *
Disallow: /admin/
Disallow: /cache/
Disallow: /converge_local/
Disallow: /hooks/
Disallow: /ips_kernel/
Disallow: /lofiversion/
Disallow: /retail/
Disallow: /public/style_captcha/
Disallow: /index.php?act=Post&CODE=02
Disallow: /index.php?act=Print&client=printer
Disallow: /index.php?app=blog&view_type=list
Disallow: /index.php?app=blog&view_type=table
Disallow: /index.php?app=blog&module=actions&section=forward&do=sendentry
Disallow: /index.php?app=blog&module=actions&section=mark&do=markallread
Disallow: /index.php?app=blog&module=display&section=blog&blogid
Disallow: /index.php?app=blog&module=display&section=top10&max_results=10&sort_key=blog_num_comments&sort_order=desc
Disallow: /index.php?app=blog&module=display&section=top10&max_results=10&sort_key=blog_num_entries&sort_order=desc
Disallow: /index.php?app=blog&module=display&section=top10&max_results=10&sort_key=blog_num_views&sort_order=desc
Disallow: /index.php?app=blog&type=all
Disallow: /index.php?app=blog&type=all&sort_key=blog_last_edate&sort_order=asc
Disallow: /index.php?app=blog&type=all&stick=1
Disallow: /index.php?app=blog&type=dash
Disallow: /index.php?app=blog&name_box=begins
Disallow: /index.php?app=calendar
Disallow: /index.php?app=calendar$
Disallow: /index.php?app=calendar&module=calendar
Disallow: /index.php?app=calendar&module=calendar
Disallow: /index.php?app=core&module=attach
Disallow: /index.php?app=core&module=task
Disallow: /index.php?app=core&module=usercp&tab=forums&area=forumsubs
Disallow: /index.php?app=core&module=usercp&tab=forums&area=watch&watch=topic
Disallow: /index.php?app=core&module=global&section=login
Disallow: /index.php?app=core&module=global&section=lostpass
Disallow: /index.php?app=core&module=global&section=register
Disallow: /index.php?app=core&module=global&section=login&do=deleteCookies
Disallow: /index.php?app=core&module=global&section=login&do=deleteCookies&k=
Disallow: /index.php?app=core&module=help
Disallow: /index.php?app=core&module=reports
Disallow: /index.php?app=core&module=reports&rcom=links
Disallow: /index.php?app=core&module=search
Disallow: /index.php?app=core&module=search&do=active
Disallow: /index.php?app=core&module=search&do=active&search_filter_app
Disallow: /index.php?app=core&module=search&do=active&search_app=ccs
Disallow: /index.php?app=core&module=search&search_in=ccs
Disallow: /index.php?app=core&module=search&search_in=forums
Disallow: /index.php?app=core&module=search&do=user_posts
Disallow: /index.php?app=core&module=search&search_in=
Disallow: /index.php?app=core&module=search&do=active&search_app=
Disallow: /index.php?app=forums&module=
Disallow: /index.php?app=forums&module=ajax&section=topics
Disallow: /index.php?app=forums&module=ajax&section=topics&do=rateTopic
Disallow: /index.php?app=forums&module=extras§ion=rating
Disallow: /index.php?app=forums&module=extras&section=forward
Disallow: /index.php?app=forums&module=extras&section=rating
Disallow: /index.php?app=forums&module=extras&section=stats
Disallow: /index.php?app=forums&module=extras&section=toggle
Disallow: /index.php?app=forums&module=extras&section=stats&do=leaders
Disallow: /index.php?app=forums&module=forums&section=attach
Disallow: /index.php?app=forums&module=forums&section=markasread
Disallow: /index.php?app=forums&module=forums&section=printtopic&client=printer
Disallow: /index.php?app=forums&module=post&section=post
Disallow: /index.php?app=forums&module=post&section=post&do=reply
Disallow: /index.php?app=forums&module=moderate
Disallow: /index.php?app=forums&module=moderate&section=moderate
Disallow: /index.php?app=gallery&module=cats&section=markread&do=category
Disallow: /index.php?app=gallery&module=post&section=comment&do=postcomment
Disallow: /index.php?app=gallery&module=stats&do=comments
Disallow: /index.php?app=gallery&module=stats&do=newest
Disallow: /index.php?app=gallery&module=stats&do=ratings
Disallow: /index.php?app=gallery&module=stats&do=views
Disallow: /index.php?app=gallery&module=user
Disallow: /index.php?app=gallery&module=user&section=user
Disallow: /index.php?app=links&module=links&section=links&do=lastsubmitted
Disallow: /index.php?app=links&module=links&section=links&do=topbyhits
Disallow: /index.php?app=links&module=links&section=links&do=topbyrating
Disallow: /index.php?app=links&module=links&section=links&do=topsubmitters
Disallow: /index.php?app=links&module=links&section=categories&do=markread&cid=
Disallow: /index.php?app=links&module=links&section=categories&do=view&cid=
Disallow: /index.php?app=links&module=links&section=links&do=markread&lid=
Disallow: /index.php?app=members&module=chat
Disallow: /index.php?app=members&module=list
Disallow: /index.php?app=members&module=messaging
Disallow: /index.php?app=members&module=online&section=online
Disallow: /index.php?app=members&module=online&sort_order=desc
Disallow: /index.php?app=members&module=online&sort_key=name&sort_order=asc&show_mem=reg
Disallow: /index.php?app=members&module=profile§ion=friends&do=remove&member_id=
Disallow: /index.php?app=members&module=profile&section=friends&do=remove&member_id=
Disallow: /index.php?app=members&module=profile&section=friends&do=add&member_id=
Disallow: /index.php?app=members&module=profile&section=friends&do=remove&member_id=
Disallow: /index.php?app=members&section=friends&module=profile&do=remove&member_id=
Disallow: /index.php?app=members&section=friends&module=profile&do=add&member_id=
Disallow: /index.php?app=members&section=friends&module=profile&do=remove&member_id=
Disallow: /index.php?app=members&module=profile&section=status&do=lockStatus
Disallow: /index.php?app=members&module=profile&section=status&do=unlockStatus
Disallow: /index.php?app=members&module=search
Disallow: /index.php?app=members&module=search&do=active
Disallow: /index.php?app=members&section=view&module=list
Disallow: /index.php?app=members&module=profile&section=friend&do=remove&member_id=
Disallow: /index.php?&unlockUserAgent=1
Disallow: /index.php?autocom=blog
Disallow: /index.php?autocom=gallery
Disallow: /index.php?automodule=blog
Disallow: /index.php?automodule=blog&blogid=
Disallow: /index.php?automodule=gallery
Disallow: /index.php?showuser=
Disallow: /index.php?sharelink=
Disallow: /*app=core&module=usercp
Disallow: /*app=core&module=usercp
Disallow: /*app=members&module=messaging
Disallow: /*&mode=linear
Disallow: /*&mode=linearplus
Disallow: /*&mode=linear
Disallow: /*&mode=linearplus
Disallow: /*&mode=threaded
Disallow: /*&st=
Disallow: /*&cat=0
Disallow: /*&p=
Disallow: /*&s=
Disallow: /*&pid=
Disallow: /*&prune_day=
Disallow: /*&hl=
Disallow: /*&st=0
Disallow: /*&show=nextnewest
Disallow: /*&start=
Disallow: /*&tag=
Disallow: /*page__cid__*$
Disallow: /*page__f__*$
Disallow: /*page__prune_day__*$
Disallow: /*page__s__*$
Disallow: /*page__show__newcomment
Disallow: /*page__sort_key__views__order_key__desc__prune_key__*
Disallow: /*page__sort_key__rating__order_key__desc__prune_key__*
Disallow: /*page__sort_key__id__order_key__desc__prune_key__*
Disallow: /*page__sort_key__comments__order_key__desc__prune_key__*
Disallow: /*page__tab__*$
Disallow: /*page__view__*$
Disallow: /*page__tag__
Disallow: /*page___gocomments
Disallow: /*prune_day*$
Disallow: /*prune__day*$
Disallow: /*view__getfirst$
Disallow: /*view__getprevious$
Disallow: /*view__getlast$
Disallow: /*view__getlastpost$
Disallow: /*view__getnewpost$
Disallow: /*view__getnext$
Disallow: /*view__new$
Disallow: /*view__old$
Disallow: /*view-getfirst$
Disallow: /*view-getprevious$
Disallow: /*view-getlast$
Disallow: /*view-getlastpost$
Disallow: /*view-getnewpost$
Disallow: /*view-getnext$
Disallow: /*view-new$
Disallow: /*view-old$
Disallow: /*&view=getlastpost$
Disallow: /*&view=getnewpost$
Disallow: /*&view=new$
Disallow: /*&view=old$
Disallow: /*&view=getfirst$
Disallow: /*&view=getlast$
Disallow: /*&view=getnext$
Disallow: /*&view=getprevious$
Disallow: /*?s=
Disallow: /blogs/*page__s__*$
Disallow: /*&cat=0

[Amfidiusz]

Tu masz duże robots.txt z community brane, z małymi zmianami moimi:

Używasz tego u siebie? Jest to sprawdzona zawartość?

[mlodszy]

Tak. Używam tego u siebie. Fakt, sporo mi blokuje, ale blokuje to co nie jest potrzebne do indeksowania i z reguły kierowało na 403.