.

ISP issues, weird topology and nmap results

<<

bobwood

Newbie
Newbie

Posts: 4

Joined: Tue Oct 11, 2011 7:20 pm

Post Tue Oct 11, 2011 8:15 pm

ISP issues, weird topology and nmap results

Hello,

I recently moved to Asia and am having some severe problems with my ISP, which they apparently are unable to solve, so I decided to investigate on my own. My connection is DSL and my ISP uses DHCP without authentication or MAC registration (as an aside, in my personal experience, this is a management and security disaster).

My subnet is x.y.128.0/17 (255.255.128.0 netmask), my gateway being x.y.128.1. For the purpose of the scan, I am not behind my usual home router.

The first thing I do not get is their network topology. Every host I nmap in my subnet is showing the same MAC address: apparently the one of the gateway. The Zenmap topology output shows a perfect star with... me (localhost) in the center... For example:

  Code:
nmap -sn --traceroute x.y.205.101

Starting Nmap 5.59BETA1 ( http://nmap.org ) at 2011-10-12 08:37 PHT
Nmap scan report for x.y.205.101.my.isp (x.y.205.101)
Host is up (0.11s latency).
MAC Address: 00:21:A0:a:b:c (Cisco Systems)

TRACEROUTE
HOP RTT       ADDRESS
1   106.86 ms x.y.205.101.my.isp (x.y.205.101)

with the MAC address always the same as that of x.y.128.1, the gateway, but the gateway's IP not showing in the route.

When I scan further, every host is showing different open ports and different OSes though, so they seem to all be real live hosts. For example:

  Code:
Nmap scan report for x.y.130.213
Host is up (0.056s latency).
Scanned at 2011-10-11 21:15:12 PHT for 30099s
Not shown: 985 filtered ports
PORT      STATE  SERVICE
687/tcp   closed asipregistry
1044/tcp  closed dcutility
1069/tcp  closed cognex-insight
1113/tcp  closed ltp-deepspace
1296/tcp  closed dproxy
1300/tcp  closed h323hostcallsc
2048/tcp  closed dls-monitor
2107/tcp  closed msmq-mgmt
2135/tcp  closed gris
2160/tcp  closed apc-2160
2602/tcp  closed ripd
4445/tcp  closed upnotifyp
5631/tcp  closed pcanywheredata
32768/tcp closed filenet-tms
50800/tcp closed unknown
MAC Address: 00:21:A0:a:b:c (Cisco Systems)

Nmap scan report for x.y.128.12
Host is up (0.20s latency).
Scanned at 2011-10-11 21:15:12 PHT for 1029s
Not shown: 990 closed ports
PORT      STATE SERVICE
135/tcp   open  msrpc
139/tcp   open  netbios-ssn
445/tcp   open  microsoft-ds
6881/tcp  open  bittorrent-tracker
45100/tcp open  unknown
49152/tcp open  unknown
49153/tcp open  unknown
49154/tcp open  unknown
49157/tcp open  unknown
49163/tcp open  unknown
MAC Address: 00:21:A0:a:b:c (Cisco Systems)

... again always with the same MAC address.

Now the problem I have is that starting from about 10AM to 7PM, I hardly can access the internet. I have found by running tcpdump and looking at the output that at these times there are packet floods at my interface, which vary from SYN, SYN/ACK and UDP packets. However these packets have a source and destination IP distinct from mine. Mostly they are SYN packets aimed at port 445 or SYN/ACK packets aimed at port 7170 and often the same IPs are involved. The source MAC is always the usual 00:21:A0:a:b:c, but the destination MAC is also not mine!! So I have no idea how and why these packets make it to my interface to start with!? All I can think of is that they have some crossed wires or short circuits in their junction boxes!

I scanned the source and destination hosts most often mentioned in the SYN/ACK flood and here is what I got:
  Code:
source:
Host is up (0.12s latency).

Not shown: 991 filtered ports

PORT     STATE  SERVICE
80/tcp   closed http

4443/tcp open   pharos

5190/tcp open   aol

5405/tcp closed pcduo

5414/tcp closed statusd

5431/tcp closed park-agent

5432/tcp closed postgresql

5440/tcp closed unknown

5566/tcp open   westec-connect

Device type: general purpose|WAP|router|firewall

Running (JUST GUESSING): Linux 2.4.X|2.6.X (99%), Netgear embedded (97%), D-Link Linux 2.4.X (95%), Netgear Linux 2.4.X (95%), HID embedded (94%), D-Link embedded (94%), Linksys embedded (94%), Peplink embedded (94%)

Aggressive OS guesses: Linux 2.4.21 - 2.4.31 (likely embedded) (99%), Linux 2.4.7 (98%), Netgear DG834GB wireless broadband router (97%), D-Link DIR-100; or Netgear KWRGR614, RT614, or WG602 router (Linux 2.4) (95%), Linux 2.4.9 (Red Hat Enterprise Linux 2.1 AS) (94%), HID EdgePlus Solo ES400 firewall (94%), D-Link DSA-3100 or Linksys WRT54GL (DD-WRT v23) WAP, or Peplink Balance 30 router (94%), Linux 2.4.20 - 2.4.27 (94%), Linux 2.6.5 (SUSE Enterprise Server 9) (94%), Linux 2.4.21 - 2.4.25 (94%)

destination:
Host is up (0.52s latency).

Not shown: 999 filtered ports

PORT    STATE  SERVICE

113/tcp closed auth

Too many fingerprints match this host to give specific OS details



Any idea, thought or  insight from people more experienced than I am would be most welcome.
<<

cd1zz

User avatar

Recruiters
Recruiters

Posts: 566

Joined: Sun Oct 03, 2010 9:01 pm

Post Tue Oct 11, 2011 9:21 pm

Re: ISP issues, weird topology and nmap results

So you're connecting your scanning box directly to the internet? It gets a public IP? Or are you scanning boxes on your own local network? How many computers are on your home network? Do you get the same results with different scanning computers?
<<

bobwood

Newbie
Newbie

Posts: 4

Joined: Tue Oct 11, 2011 7:20 pm

Post Wed Oct 12, 2011 4:59 am

Re: ISP issues, weird topology and nmap results

Yes for the purpose of this scan I did connect my scanning box straight to the internet. One of the reasons is that I was interested in neighbouring MAC addresses, which I can only get at if  my scan box is directly on the same subnet as the machines of interest. In this config my scan box is the only host at home connected to the net, through my dsl modem configured as a bridge (not a router or DHCP server). In my normal setup, I have a Linksys router loaded with dd-wrt sitting between the modem and my local home network.

Yes the scan box does get a public IP (from the ISP's DHCP server) in the x.y.128.0/17 subnet mentioned in OP. To be clear here is what the vlan2 of my dd-wrt router looks like, and my scan box eth0 looks the same when it is connected directly:

  Code:
vlan2 Link encap:Ethernet  HWaddr 68:7F:74:h:i:j
        inet addr:x.y.168.22  Bcast:x.y.255.255  Mask:255.255.128.0
        UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
        RX packets:48901096 errors:0 dropped:0 overruns:0 frame:0
        TX packets:103097 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:0
        RX bytes:2624846509 (2.4 GiB)  TX bytes:15942511 (15.2 MiB)


I did not do the nmap scan with a different computer, however I did the tcpdump with both my scan box and also from the command line of my dd-wrt Linksys router. Results are the same: SYN, SYN/ACK or UDP packet flood with src and dst IP's and MAC's having nothing to do with me. See http://imageshack.us/photo/my-images/37/tcpdump.png/, screenshot of the pcap file in wireshark. It goes on like that for hours. In the packet flood, the MAC address of the dst machine is 00:23:69:a:b:c, but if I nmap that IP, it resolves again to 00:21:a0:d:e:f, just as every other IP on the subnet.

Outside the hours mentioned in OP, traffic is as normal: next to zero bandwidth when my computer is idle, and then I only see normal packets meant for my IP/MAC (or from my IP/MAC) at the interface.

I even tried to change the dsl modem for a different model and brand, same result. This thing only started happening when I came back from holiday one week ago. Before that everything was more or less normal apart from occasional intermittent connectivity issues and sluggish connection at rush hours.

Whichever way, I may lack experience but I can't understand how I can be only a single hop away from every host on the subnet, and how every IP on the subnet resolves to a single MAC!? Is this normal??
Last edited by bobwood on Wed Oct 12, 2011 5:29 am, edited 1 time in total.
<<

cd1zz

User avatar

Recruiters
Recruiters

Posts: 566

Joined: Sun Oct 03, 2010 9:01 pm

Post Wed Oct 12, 2011 8:23 am

Re: ISP issues, weird topology and nmap results

It almost sounds like there is a verbose switch port setup somewhere. Sounds like you're getting other people's traffic?

What does the output of your local arp table look like?
<<

sil

User avatar

Hero Member
Hero Member

Posts: 551

Joined: Thu Mar 20, 2008 8:01 am

Location: ::1

Post Wed Oct 12, 2011 8:31 am

Re: ISP issues, weird topology and nmap results

First, let's break down how you are a single hop away from everyone else on your network, then I will try to explain to you why no username/auth is not necessarily the end of days.

When you connect to your provider, just like everyone else you're likely connected to a DSLAM unless I misread a) DSL and or b) your provider is doing something strange. A DSLAM is nothing more than a layer 2 switch. It does no routing that is handled by BRAS (remote access servers).

Topology:

  Code:
You --> DSLAM --> BRAS --> Router --> Internet
Your Neighbor --> DSLAM --> BRAS --> Router --> Internet
Someone down the road --> DSLAM --> BRAS --> Router --> Internet
etc. --> DSLAM --> BRAS --> Router --> Internet
etc. --> DSLAM --> BRAS --> Router --> Internet


With the topology out of the way, take note you're in a similar set-up to a LAN. In this case, its a MAN/WAN depending on your ISP. What you're likely seeing is multicasted garbage. Networks talk a lot and often you'll see things that don't make sense (tons of ports opened, etc.) If you needed/wanted to you could put a tap to really figure out what is going on but depending on your TOS/SLA and legal bindings where you're located, you may or may not be able to do so. In order to TRULY see what it going on outside of the garbage you're seeing, a tap on a DSL environment would be configured as follows:

  Code:
Your PC --> Modem --> DSL Line --> TAP --> DSL Line --> ISP


Your tap would be positioned at your dropoff before it interconnects with your home/apartment, etc. Neither here nor there, but you asked.

So in your initial comment: "My connection is DSL and my ISP uses DHCP without authentication or MAC registration (as an aside, in my personal experience, this is a management and security disaster)." Your personal experience and your understanding of what is going on are two different things. You "assume" there is no authentication going on.

Most ISPs that provide DSL will likely be using RADIUS for authentication (some *might* use DIAMETER). Some use PPP, some PPPoE and so on and so forth. RADIUS does not transmit passwords but hashes. It is likely that your provider is authenticating based on perhaps your CALLERID, your line card porton the DSLAM, modem identifier other than MAC, a hashed MAC, and or any combination of those. That information will be stored in RADIUS AVP types 1 and 2 however, you would never see that information clear text as its hashed. Could be an MD5 hash, SHA hash, could be just a hexcode hash.

Now on to your original problem: "starting from about 10AM to 7PM, I hardly can access the internet." This sounds like you're on a DSLAM that is oversubscribed. Period. Golf ball through the water hose effect. Nothing more, nothing less, nothing sinister. Depending on where in ASIA you are, you might also go through extreme filtering (Great Firewall). Think about the topology I explained (DSLAM) you're on a switch nothing more that connects elsewhere. WHAT that connects to is what's important. For example, imagine the following:

  Code:
DSLAM --> BRAS --> Router --> DS3 --> Internet


A DS3 will only get you 44.(5-7)Mb. If your provider didn't properly allocate/traffic shape those line cards, one person could be monopolizing the bandwidth OR, if they have too many people on the DSLAM, they're saturating it. If your provider states your download speed should be say 5Mb, then that should be your speed. However, you have to factor how many people would be told the same: "Well give you 5Mb downloads and 1.5 uploads" now they place 10 people on the circuit and when in use, traffic s 50Mb, much more than a 44Mb connection can handle.

When providers initially shaped/diagrammed many of these services (DSL in the 90s), the thinking was that no one would be online 24x7x365. Nowadays people don't shut down their networks/equipment. So if they filled up their DSLAM and circuit, you're screwed. Nothing nefarious about what you mentioned. Just seems like your provider is hosed.
<<

bobwood

Newbie
Newbie

Posts: 4

Joined: Tue Oct 11, 2011 7:20 pm

Post Wed Oct 12, 2011 7:49 pm

Re: ISP issues, weird topology and nmap results

Here is the output of the arp cache of my dd-wrt router (just after pinging x.y.152.12):
  Code:
 arp -a
MyBox (192.168.1.112) at 00:22:43:e:f:g [ether]  on br0
x.y.128.1.My.ISP (x.y.128.1) at 00:21:A0:a:b:c [ether]  on vlan2
x.y.152.12.My.ISP (x.y.152.12) at 00:21:A0:a:b:c [ether]  on vlan2

Again, I am not sure why x.y.152.12 maps to the same MAC address as x.y.128.1. Even with the DSLAM explanation above: if the DSLAM acts as a switch, then I should be getting the actual MAC address of x.y.152.12, should I not?

I don't know what is a verbose switch port?

I hear you on the topology, although I did other tests at a different time, and it seems to be routing through x.y.128.1 now!?:
  Code:
traceroute x.y.152.12
traceroute to x.y.152.12 (x.y.152.12), 30 hops max, 38 byte packets
 1  x.y.128.1.My.ISP (x.y.128.1)  57.862 ms  29.035 ms  28.494 ms
 2  * * *

(if you have an idea of commands I could try to confirm, please let me know).

I also hear you on the "hidden authentication".

And thank you for your explanations, this is most helpful. I know a little because I administer a small office network and networking is a hobby for me, but I have no experience on how things are done on the ISP side.

I beg to disagree on "your DSLAM is just oversubscribed" though. I have bandwidth graphs on my dd-wrt router, and it's not like traffic is gently creeping up at rush hour. It suddenly goes from zero to maxed out, unsolicited, and what I see on my tcpdump (confirmed many times) is the SYN or SYN/ACK packet flood, with (non multicast/broadcast) IP's and MAC addresses that are not mines. Look at the wireshark screenshot. At normal times, the only traffic at my interface is (as I would expect) traffic to and from my IP/MAC and the occasional broadcast. Again, this only started happening recently. Before that, my "1Mbps" connection would give me speeds between 500 and 700 Kbps depending on the time of the day.

I am also not sure I understand your idea with the tap. What could be happening between the tap and the LAN side of the modem that could cause this flood? This cannot be caused by something on my side (like a virus): this is happening with ot without my dd-wrt router in line, and it is happening both when I am logged into Linux and Windows with my scan box, and also happening when I use a different laptop. These rogue packets are definitely coming from outside, not originating from my side.

Also, there is no filtering in this country, the net is wide open.

It has been a week that I have been trying to get my ISP to fix this. If it is not done today, I switch to their competition.

I am still curious how it can happen that I am getting traffic at my interface that is not for me. I mean if the DSLAM you mentioned acts as a switch, this should not happen right? And if it acted as a hub, I would see other's traffic at all times, not just when the flood happens.

If anyone has ideas of commands I could run to further document this, please let me know.
Last edited by bobwood on Wed Oct 12, 2011 9:10 pm, edited 1 time in total.
<<

bobwood

Newbie
Newbie

Posts: 4

Joined: Tue Oct 11, 2011 7:20 pm

Post Thu Oct 13, 2011 5:02 am

Re: ISP issues, weird topology and nmap results

I just tracerouted one of the most common target IP's of the packet flood while there was no flood at my interface, and I got some pretty crazy result:
112.201.130.84
  Code:
 traceroute x.y.130.84
traceroute to x.y.130.84 (x.y.130.84), 30 hops max, 38 byte packets
 1  x.y.128.1.my.isp (x.y.128.1)  62.841 ms  28.164 ms  28.448 ms
 2  * * x.y.187.26.my.isp (x.y.187.26)  55.498 ms
 3  * x.y.128.1.my.isp (x.y.128.1)  64.130 ms  60.648 ms
 4  x.y.187.26.my.isp (x.y.187.26)  90.112 ms  87.804 ms  89.228 ms
 5  x.y.128.1.my.isp (x.y.128.1)  98.640 ms  96.709 ms  93.321 ms
 6  x.y.187.26.my.isp (x.y.187.26)  128.730 ms  123.274 ms  125.452 ms
 7  x.y.128.1.my.isp (x.y.128.1)  134.578 ms  128.676 ms  133.660 ms
 8  x.y.187.26.my.isp (x.y.187.26)  159.615 ms  164.823 ms  163.425 ms
 9  x.y.128.1.my.isp (x.y.128.1)  175.796 ms  172.736 ms  170.371 ms
10  x.y.187.26.my.isp (x.y.187.26)  198.230 ms x.y.251.47.my.isp (x.y.251.47)  193.421 ms x.y.187.26.my.isp (x.y.187.26)  194.419 ms
11  x.y.128.1.my.isp (x.y.128.1)  202.865 ms  199.893 ms  203.395 ms
12  x.y.187.26.my.isp (x.y.187.26)  238.256 ms  231.051 ms x.y.251.47.my.isp (x.y.251.47)  242.889 ms
13  x.y.128.1.my.isp (x.y.128.1)  234.415 ms  239.118 ms  244.420 ms
14  x.y.187.26.my.isp (x.y.187.26)  268.483 ms x.y.218.35.my.isp (x.y.218.35)  266.568 ms x.y.187.26.my.isp (x.y.187.26)  263.414 ms
15  x.y.128.1.my.isp (x.y.128.1)  264.052 ms  291.715 ms  273.907 ms
16  x.y.187.26.my.isp (x.y.187.26)  300.505 ms  296.105 ms  302.381 ms
17  x.y.128.1.my.isp (x.y.128.1)  298.706 ms  312.134 ms  310.636 ms
18  x.y.187.26.my.isp (x.y.187.26)  343.079 ms  330.668 ms  335.475 ms
19  x.y.128.1.my.isp (x.y.128.1)  349.488 ms  339.655 ms  348.008 ms
20  x.y.187.26.my.isp (x.y.187.26)  362.616 ms  370.036 ms  378.598 ms
21  x.y.128.1.my.isp (x.y.128.1)  371.539 ms  384.478 ms  383.735 ms
22  x.y.218.35.my.isp (x.y.218.35)  414.256 ms x.y.187.26.my.isp (x.y.187.26)  415.995 ms x.y.251.47.my.isp (x.y.251.47)  400.182 ms
23  x.y.128.1.my.isp (x.y.128.1)  396.818 ms * *
24  * * *
25  *

???

I also tracerouted it earlier on while the flood was on at my interface, and I found my own IP in the route several times as well. Forgot to save the output though.

I don't know could it be that someone or some virus on the subnet is playing around and poisoning the arp cache of their router or whatever?

What do ISP's do to prevent this kind of attack if everyone is on the same giant switch (DSLAM)? I mean here they cannot be using static arp entries since they do not force the users to register their MAC, right?
<<

WCNA

User avatar

Full Member
Full Member

Posts: 187

Joined: Wed Mar 02, 2011 8:05 am

Location: Florida

Post Thu Oct 13, 2011 11:49 am

Re: ISP issues, weird topology and nmap results

Back when dsl was just coming out, I was an installer for the phone company. I don't know whether there was additional identification or not going on but I ran the wire in the CO to the dslam and created a path through the cross-connects directly to your house. No security was needed as far as that was concerned because if you didn't pay your bill we'd just untie your line in the CO and you would be gone. I don't know what went on after the dslam as that was not part of my job.

In addition, DSL is very susceptible to interference on the wire. It's possible you don't have a clean line and at a certain time of day, you're getting crapped on by some issue. As sil pointed out, it could oversubscription as well. ISPs work on an oversubscription model because service would be more expensive if everyone used all of their bandwidth all the time. The ISP I work for monitors how much bandwidth is used overall and at what times in order to determine how much to charge (and how much to limit) for your bulk connection.

It's also possible if you are on a switched network, i.e. the same broadcast domain as someone with a virus (or many with a virus), you could be getting slammed with crap. As sil suggested, the way to test this is with a packet capture before and during the problem so you can analyze what traffic you're seeing and what it is doing.
Last edited by WCNA on Thu Oct 13, 2011 12:01 pm, edited 1 time in total.
ISC2 Associate, WCNA, CWNA, OSCP, Network+
<<

WCNA

User avatar

Full Member
Full Member

Posts: 187

Joined: Wed Mar 02, 2011 8:05 am

Location: Florida

Post Sat Oct 15, 2011 4:59 pm

Re: ISP issues, weird topology and nmap results

One more thing and then I'll shut up.

I mentioned in another thread I'm reading up on wireless lan controllers because that's what companies seem to want WLAN engineers to know. Cisco uses LWAPP and now CAPWAP. I wanted to see what my new favorite product (UBNT's Unifi) uses- capwap or lwapp.

It turns out it's neither- it uses TR-069...the same protocol that DSL modems use. AHHHH! So now after I read these thousand pages from Cisco, I have to read up on yet-another-protocol (in addition to doing the SWSE at the same time and working with the networksims stuff and a full time job).

I'm already seeing crosseyed but I gotta admit, I'm an info junkie and it is a lot of fun....complicated as hell but a lotta fun. Oh yeah, maybe then I'll get to the VMware security course I won.

Anyway bob, maybe this will help.
http://www.breakingpointsystems.com/com ... n-toolkit/
Last edited by WCNA on Sat Oct 15, 2011 5:53 pm, edited 1 time in total.
ISC2 Associate, WCNA, CWNA, OSCP, Network+

Return to Networking

Who is online

Users browsing this forum: No registered users and 1 guest

.
Powered by phpBB® Forum Software © phpBB Group.
Designed by ST Software