1 Network Working Group                                           A. Kumar   
    2 Request for Comments: 1536                                     J. Postel   
    3 Category: Informational                                        C. Neuman   
    4                                                                      ISI   
    5                                                                P. Danzig   
    6                                                                S. Miller   
    7                                                                      USC   
    8                                                             October 1993   
   11           Common DNS Implementation Errors and Suggested Fixes             
   13 Status of this Memo                                                        
   15    This memo provides information for the Internet community.  It does     
   16    not specify an Internet standard.  Distribution of this memo is         
   17    unlimited.                                                              
   19 Abstract                                                                   
   21    This memo describes common errors seen in DNS implementations and       
   22    suggests some fixes. Where applicable, violations of recommendations    
   23    from STD 13, RFC 1034 and STD 13, RFC 1035 are mentioned. The memo      
   24    also describes, where relevant, the algorithms followed in BIND         
   25    (versions 4.8.3 and 4.9 which the authors referred to) to serve as an   
   26    example.                                                                
   28 Introduction                                                               
   30    The last few years have seen, virtually, an explosion of DNS traffic    
   31    on the NSFnet backbone. Various DNS implementations and various         
   32    versions of these implementations interact with each other, producing   
   33    huge amounts of unnecessary traffic. Attempts are being made by         
   34    researchers all over the internet, to document the nature of these      
   35    interactions, the symptomatic traffic patterns and to devise remedies   
   36    for the sick pieces of software.                                        
   38    This draft is an attempt to document fixes for known DNS problems so    
   39    people know what problems to watch out for and how to repair broken     
   40    software.                                                               
   42 1. Fast Retransmissions                                                    
   44    DNS implements the classic request-response scheme of client-server     

The IETF is responsible for the creation and maintenance of the DNS RFCs. The ICANN DNS RFC annotation project provides a forum for collecting community annotations on these RFCs as an aid to understanding for implementers and any interested parties. The annotations displayed here are not the result of the IETF consensus process.

This RFC is included in the DNS RFCs annotation project whose home page is here.

   45    interaction. UDP is, therefore, the chosen protocol for communication   
   46    though TCP is used for zone transfers. The onus of requerying in case   
   47    no response is seen in a "reasonable" period of time, lies with the     
   48    client. Although RFC 1034 and 1035 do not recommend any                 
   52 Kumar, Postel, Neuman, Danzig & Miller                          [Page 1]   

   53 RFC 1536            Common DNS Implementation Errors        October 1993   
   56    retransmission policy, RFC 1035 does recommend that the resolvers       
   57    should cycle through a list of servers. Both name servers and stub      
   58    resolvers should, therefore, implement some kind of a retransmission    
   59    policy based on round trip time estimates of the name servers. The      
   60    client should back-off exponentially, probably to a maximum timeout     
   61    value.                                                                  
   63    However, clients might not implement either of the two. They might      
   64    not wait a sufficient amount of time before retransmitting or they      
   65    might not back-off their inter-query times sufficiently.                
   67    Thus, what the server would see will be a series of queries from the    
   68    same querying entity, spaced very close together. Of course, a          
   69    correctly implemented server discards all duplicate queries but the     
   70    queries contribute to wide-area traffic, nevertheless.                  
   72    We classify a retransmission of a query as a pure Fast retry timeout    
   73    problem when a series of query packets meet the following conditions.   
   75       a. Query packets are seen within a time less than a "reasonable      
   76          waiting period" of each other.                                    
   78       b. No response to the original query was seen i.e., we see two or    
   79          more queries, back to back.                                       
   81       c. The query packets share the same query identifier.                
   83       d. The server eventually responds to the query.                      
   85 A GOOD IMPLEMENTATION:                                                     
   87    BIND (we looked at versions 4.8.3 and 4.9) implements a good            
   88    retransmission algorithm which solves or limits all of these            
   89    problems.  The Berkeley stub-resolver queries servers at an interval    
   90    that starts at the greater of 4 seconds and 5 seconds divided by the    
   91    number of servers the resolver queries. The resolver cycles through     
   92    servers and at the end of a cycle, backs off the time out               
   93    exponentially.                                                          
   95    The Berkeley full-service resolver (built in with the program           
   96    "named") starts with a time-out equal to the greater of 4 seconds and   
   97    two times the round-trip time estimate of the server.  The time-out     
   98    is backed off with each cycle, exponentially, to a ceiling value of     
   99    45 seconds.                                                             
  107 Kumar, Postel, Neuman, Danzig & Miller                          [Page 2]   

  108 RFC 1536            Common DNS Implementation Errors        October 1993   
  111 FIXES:                                                                     
  113       a. Estimate round-trip times or set a reasonably high initial        
  114          time-out.                                                         
  116       b. Back-off timeout periods exponentially.                           
  118       c. Yet another fundamental though difficult fix is to send the       
  119          client an acknowledgement of a query, with a round-trip time      
  120          estimate.                                                         
  122    Since UDP is used, no response is expected by the client until the      
  123    query is complete.  Thus, it is less likely to have information about   
  124    previous packets on which to estimate its back-off time.  Unless, you   
  125    maintain state across queries, so subsequent queries to the same        
  126    server use information from previous queries.  Unfortunately, such      
  127    estimates are likely to be inaccurate for chained requests since the    
  128    variance is likely to be high.                                          
  130    The fix chosen in the ARDP library used by Prospero is that the         
  131    server will send an initial acknowledgement to the client in those      
  132    cases where the server expects the query to take a long time (as        
  133    might be the case for chained queries).  This initial acknowledgement   
  134    can include an expected time to wait before retrying.                   
  136    This fix is more difficult since it requires that the client software   
  137    also be trained to expect the acknowledgement packet. This, in an       
  138    internet of millions of hosts is at best a hard problem.                
  140 2. Recursion Bugs                                                          
  142    When a server receives a client request, it first looks up its zone     
  143    data and the cache to check if the query can be answered. If the        
  144    answer is unavailable in either place, the server seeks names of        
  145    servers that are more likely to have the information, in its cache or   
  146    zone data. It then does one of two things. If the client desires the    
  147    server to recurse and the server architecture allows recursion, the     
  148    server chains this request to these known servers closest to the        
  149    queried name. If the client doesn't seek recursion or if the server     
  150    cannot handle recursion, it returns the list of name servers to the     
  151    client assuming the client knows what to do with these records.         
  153    The client queries this new list of name servers to get either the      
  154    answer, or names of another set of name servers to query. This          
  155    process repeats until the client is satisfied. Servers might also go    
  156    through this chaining process if the server returns a CNAME record      
  157    for the queried name. Some servers reprocess this name to try and get   
  158    the desired record type.                                                
  162 Kumar, Postel, Neuman, Danzig & Miller                          [Page 3]   

  163 RFC 1536            Common DNS Implementation Errors        October 1993   
  166    However, in certain cases, this chain of events may not be good. For    
  167    example, a broken or malicious name server might list itself as one     
  168    of the name servers to query again. The unsuspecting client resends     
  169    the same query to the same server.                                      
  171    In another situation, more difficult to detect, a set of servers        
  172    might form a loop wherein A refers to B and B refers to A. This loop    
  173    might involve more than two servers.                                    
  175    Yet another error is where the client does not know how to process      
  176    the list of name servers returned, and requeries the same server        
  177    since that is one (of the few) servers it knows.                        
  179    We, therefore, classify recursion bugs into three distinct              
  180    categories:                                                             
  182       a. Ignored referral: Client did not know how to handle NS records    
  183          in the AUTHORITY section.                                         
  185       b. Too many referrals: Client called on a server too many times,     
  186          beyond a "reasonable" number, with same query. This is            
  187          different from a Fast retransmission problem and a Server         
  188          Failure detection problem in that a response is seen for every    
  189          query.  Also, the identifiers are always different. It implies    
  190          client is in a loop and should have detected that and broken      
  191          it.  (RFC 1035 mentions that client should not recurse beyond     
  192          a certain depth.)                                                 
  194       c. Malicious Server: a server refers to itself in the authority      
  195          section. If a server does not have an answer now, it is very      
  196          unlikely it will be any better the next time you query it,        
  197          specially when it claims to be authoritative over a domain.       
  199       RFC 1034 warns against such situations, on page 35.                  
  201       "Bound the amount of work (packets sent, parallel processes          
  202        started) so that a request can't get into an infinite loop or       
  203        start off a chain reaction of requests or queries with other        
  204        implementations EVEN IF SOMEONE HAS INCORRECTLY CONFIGURED          
  205        SOME DATA."                                                         
  207 A GOOD IMPLEMENTATION:                                                     
  209    BIND fixes at least one of these problems. It places an upper limit     
  210    on the number of recursive queries it will make, to answer a            
  211    question.  It chases a maximum of 20 referral links and 8 canonical     
  212    name translations.                                                      
  217 Kumar, Postel, Neuman, Danzig & Miller                          [Page 4]   

  218 RFC 1536            Common DNS Implementation Errors        October 1993   
  221 FIXES:                                                                     
  223       a. Set an upper limit on the number of referral links and CNAME      
  224          links you are willing to chase.                                   
  226          Note that this is not guaranteed to break only recursion loops.   
  227          It could, in a rare case, prune off a very long search path,      
  228          prematurely.  We know, however, with high probability, that if    
  229          the number of links cross a certain metric (two times the depth   
  230          of the DNS tree), it is a recursion problem.                      
  232       b. Watch out for self-referring servers. Avoid them whenever         
  233          possible.                                                         
  235       c. Make sure you never pass off an authority NS record with your     
  236          own name on it!                                                   
  238       d. Fix clients to accept iterative answers from servers not built    
  239          to provide recursion. Such clients should either be happy with    
  240          the non-authoritative answer or be willing to chase the           
  241          referral links themselves.                                        
  243 3. Zero Answer Bugs:                                                       
  245    Name servers sometimes return an authoritative NOERROR with no          
  246    ANSWER, AUTHORITY or ADDITIONAL records. This happens when the          
  247    queried name is valid but it does not have a record of the desired      
  248    type. Of course, the server has authority over the domain.              
  250    However, once again, some implementations of resolvers do not           
  251    interpret this kind of a response reasonably. They always expect an     
  252    answer record when they see an authoritative NOERROR. These entities    
  253    continue to resend their queries, possibly endlessly.                   
  255 A GOOD IMPLEMENTATION                                                      
  257    BIND resolver code does not query a server more than 3 times. If it     
  258    is unable to get an answer from 4 servers, querying them three times    
  259    each, it returns error.                                                 
  261    Of course, it treats a zero-answer response the way it should be        
  262    treated; with respect!                                                  
  264 FIXES:                                                                     
  266       a. Set an upper limit on the number of retransmissions for a given   
  267          query, at the very least.                                         
  272 Kumar, Postel, Neuman, Danzig & Miller                          [Page 5]   

  273 RFC 1536            Common DNS Implementation Errors        October 1993   
  276       b. Fix resolvers to interpret such a response as an authoritative    
  277          statement of non-existence of the record type for the given       
  278          name.                                                             
  280 4. Inability to detect server failure:                                     
  282    Servers in the internet are not very reliable (they go down every       
  283    once in a while) and resolvers are expected to adapt to the changed     
  284    scenario by not querying the server for a while. Thus, when a server    
  285    does not respond to a query, resolvers should try another server.       
  286    Also, non-stub resolvers should update their round trip time estimate   
  287    for the server to a large value so that server is not tried again       
  288    before other, faster servers.                                           
  290    Stub resolvers, however, cycle through a fixed set of servers and if,   
  291    unfortunately, a server is down while others do not respond for other   
  292    reasons (high load, recursive resolution of query is taking more time   
  293    than the resolver's time-out, ....), the resolver queries the dead      
  294    server again! In fact, some resolvers might not set an upper limit on   
  295    the number of query retransmissions they will send and continue to      
  296    query dead servers indefinitely.                                        
  298    Name servers running system or chained queries might also suffer from   
  299    the same problem. They store names of servers they should query for a   
  300    given domain. They cycle through these names and in case none of them   
  301    answers, hit each one more than one. It is, once again, important       
  302    that there be an upper limit on the number of retransmissions, to       
  303    prevent network overload.                                               
  305    This behavior is clearly in violation of the dictum in RFC 1035 (page   
  306    46)                                                                     
  308       "If a resolver gets a server error or other bizarre response         
  309        from a name server, it should remove it from SLIST, and may         
  310        wish to schedule an immediate transmission to the next              
  311        candidate server address."                                          
  313    Removal from SLIST implies that the server is not queried again for     
  314    some time.                                                              
  316    Correctly implemented full-service resolvers should, as pointed out     
  317    before, update round trip time values for servers that do not respond   
  318    and query them only after other, good servers. Full-service resolvers   
  319    might, however, not follow any of these common sense directives. They   
  320    query dead servers, and they query them endlessly.                      
  327 Kumar, Postel, Neuman, Danzig & Miller                          [Page 6]   

  328 RFC 1536            Common DNS Implementation Errors        October 1993   
  331 A GOOD IMPLEMENTATION:                                                     
  333    BIND places an upper limit on the number of times it queries a          
  334    server.  Both the stub-resolver and the full-service resolver code do   
  335    this.  Also, since the full-service resolver estimates round-trip       
  336    times and sorts name server addresses by these estimates, it does not   
  337    query a dead server again, until and unless all the other servers in    
  338    the list are dead too!  Further, BIND implements exponential back-off   
  339    too.                                                                    
  341 FIXES:                                                                     
  343       a. Set an upper limit on number of retransmissions.                  
  345       b. Measure round-trip time from servers (some estimate is better     
  346          than none). Treat no response as a "very large" round-trip        
  347          time.                                                             
  349       c. Maintain a weighted rtt estimate and decay the "large" value      
  350          slowly, with time, so that the server is eventually tested        
  351          again, but not after an indefinitely long period.                 
  353       d. Follow an exponential back-off scheme so that even if you do      
  354          not restrict the number of queries, you do not overload the       
  355          net excessively.                                                  
  357 5. Cache Leaks:                                                            
  359    Every resource record returned by a server is cached for TTL seconds,   
  360    where the TTL value is returned with the RR. Full-service (or stub)     
  361    resolvers cache the RR and answer any queries based on this cached      
  362    information, in the future, until the TTL expires. After that, one      
  363    more query to the wide-area network gets the RR in cache again.         
  365    Full-service resolvers might not implement this caching mechanism       
  366    well. They might impose a limit on the cache size or might not          
  367    interpret the TTL value correctly. In either case, queries repeated     
  368    within a TTL period of a RR constitute a cache leak.                    
  370 A GOOD/BAD IMPLEMENTATION:                                                 
  372    BIND has no restriction on the cache size and the size is governed by   
  373    the limits on the virtual address space of the machine it is running    
  374    on. BIND caches RRs for the duration of the TTL returned with each      
  375    record.                                                                 
  377    It does, however, not follow the RFCs with respect to interpretation    
  378    of a 0 TTL value. If a record has a TTL value of 0 seconds, BIND uses   
  382 Kumar, Postel, Neuman, Danzig & Miller                          [Page 7]   

  383 RFC 1536            Common DNS Implementation Errors        October 1993   
  386    the minimum TTL value, for that zone, from the SOA record and caches    
  387    it for that duration. This, though it saves some traffic on the         
  388    wide-area network, is not correct behavior.                             
  390 FIXES:                                                                     
  392       a. Look over your caching mechanism to ensure TTLs are interpreted   
  393          correctly.                                                        
  395       b. Do not restrict cache sizes (come on, memory is cheap!).          
  396          Expired entries are reclaimed periodically, anyway. Of course,    
  397          the cache size is bound to have some physical limit. But, when    
  398          possible, this limit should be large (run your name server on     
  399          a machine with a large amount of physical memory).                
  401       c. Possibly, a mechanism is needed to flush the cache, when it is    
  402          known or even suspected that the information has changed.         
  404 6. Name Error Bugs:                                                        
  406    This bug is very similar to the Zero Answer bug. A server returns an    
  407    authoritative NXDOMAIN when the queried name is known to be bad, by     
  408    the server authoritative for the domain, in the absence of negative     
  409    caching. This authoritative NXDOMAIN response is usually accompanied    
  410    by the SOA record for the domain, in the authority section.             
  412    Resolvers should recognize that the name they queried for was a bad     
  413    name and should stop querying further.                                  
  415    Some resolvers might, however, not interpret this correctly and         
  416    continue to query servers, expecting an answer record.                  
  418    Some applications, in fact, prompt NXDOMAIN answers! When given a       
  419    perfectly good name to resolve, they append the local domain to it      
  420    e.g., an application in the domain "foo.bar.com", when trying to        
  421    resolve the name "usc.edu" first tries "usc.edu.foo.bar.com", then      
  422    "usc.edu.bar.com" and finally the good name "usc.edu". This causes at   
  423    least two queries that return NXDOMAIN, for every good query. The       
  424    problem is aggravated since the negative answers from the previous      
  425    queries are not cached.  When the same name is sought again, the        
  426    process repeats.                                                        
  428    Some DNS resolver implementations suffer from this problem, too. They   
  429    append successive sub-parts of the local domain using an implicit       
  430    searchlist mechanism, when certain conditions are satisfied and try     
  431    the original name, only when this first set of iterations fails. This   
  432    behavior recently caused pandemonium in the Internet when the domain    
  433    "edu.com" was registered and a wildcard "CNAME" record placed at the    
  437 Kumar, Postel, Neuman, Danzig & Miller                          [Page 8]   

  438 RFC 1536            Common DNS Implementation Errors        October 1993   
  441    top level. All machines from "com" domains trying to connect to hosts   
  442    in the "edu" domain ended up with connections to the local machine in   
  443    the "edu.com" domain!                                                   
  445 GOOD/BAD IMPLEMENTATIONS:                                                  
  447    Some local versions of BIND already implement negative caching. They    
  448    typically cache negative answers with a very small TTL, sufficient to   
  449    answer a burst of queries spaced close together, as is typically        
  450    seen.                                                                   
  452    The next official public release of BIND (4.9.2) will have negative     
  453    caching as an ifdef'd feature.                                          
  455    The BIND resolver appends local domain to the given name, when one of   
  456    two conditions is met:                                                  
  458       i.  The name has no periods and the flag RES_DEFNAME is set.         
  459       ii. There is no trailing period and the flag RES_DNSRCH is set.      
  461    The flags RES_DEFNAME and RES_DNSRCH are default resolver options, in   
  462    BIND, but can be changed at compile time.                               
  464    Only if the name, so generated, returns an NXDOMAIN is the original     
  465    name tried as a Fully Qualified Domain Name. And only if it contains    
  466    at least one period.                                                    
  468 FIXES:                                                                     
  470       a. Fix the resolver code.                                            
  472       b. Negative Caching. Negative caching servers will restrict the      
  473          traffic seen on the wide-area network, even if not curb it        
  474          altogether.                                                       
  476       c. Applications and resolvers should not append the local domain to  
  477          names they seek to resolve, as far as possible. Names             
  478          interspersed with periods should be treated as Fully Qualified    
  479          Domain Names.                                                     
  481          In other words, Use searchlists only when explicitly specified.   
  482          No implicit searchlists should be used. A name that contains      
  483          any dots should first be tried as a FQDN and if that fails, with  
  484          the local domain name (or searchlist if specified) appended. A    
  485          name containing no dots can be appended with the searchlist right 
  486          away, but once again, no implicit searchlists should be used.     
  492 Kumar, Postel, Neuman, Danzig & Miller                          [Page 9]   

  493 RFC 1536            Common DNS Implementation Errors        October 1993   
  496    Associated with the name error bug is another problem where a server    
  497    might return an authoritative NXDOMAIN, although the name is valid. A   
  498    secondary server, on start-up, reads the zone information from the      
  499    primary, through a zone transfer. While it is in the process of         
  500    loading the zones, it does not have information about them, although    
  501    it is authoritative for them.  Thus, any query for a name in that       
  502    domain is answered with an NXDOMAIN response code. This problem might   
  503    not be disastrous were it not for negative caching servers that cache   
  504    this answer and so propagate incorrect information over the internet.   
  506 BAD IMPLEMENTATION:                                                        
  508    BIND apparently suffers from this problem.                              
  510    Also, a new name added to the primary database will take a while to     
  511    propagate to the secondaries. Until that time, they will return         
  512    NXDOMAIN answers for a good name. Negative caching servers store this   
  513    answer, too and aggravate this problem further. This is probably a      
  514    more general DNS problem but is apparently more harmful in this         
  515    situation.                                                              
  517 FIX:                                                                       
  519       a. Servers should start answering only after loading all the zone    
  520          data. A failed server is better than a server handing out         
  521          incorrect information.                                            
  523       b. Negative cache records for a very small time, sufficient only     
  524          to ward off a burst of requests for the same bad name. This       
  525          could be related to the round-trip time of the server from        
  526          which the negative answer was received. Alternatively, a          
  527          statistical measure of the amount of time for which queries       
  528          for such names are received could be used. Minimum TTL value      
  529          from the SOA record is not advisable since they tend to be        
  530          pretty large.                                                     
  532       c. A "PUSH" (or, at least, a "NOTIFY") mechanism should be allowed   
  533          and implemented, to allow the primary server to inform            
  534          secondaries that the database has been modified since it last     
  535          transferred zone data.  To alleviate the problem of "too many     
  536          zone transfers" that this might cause, Incremental Zone           
  537          Transfers should also be part of DNS.  Also, the primary should   
  538          not NOTIFY/PUSH with every update but bunch a good number         
  539          together.                                                         
  547 Kumar, Postel, Neuman, Danzig & Miller                         [Page 10]   

  548 RFC 1536            Common DNS Implementation Errors        October 1993   
  551 7. Format Errors:                                                          
  553    Some resolvers issue query packets that do not necessarily conform to   
  554    standards as laid out in the relevant RFCs. This unnecessarily          
  555    increases net traffic and wastes server time.                           
  557 FIXES:                                                                     
  559       a. Fix resolvers.                                                    
  561       b. Each resolver verify format of packets before sending them out,   
  562          using a mechanism outside of the resolver. This is, obviously,    
  563          needed only if step 1 cannot be followed.                         
  565 References                                                                 
  567    [1] Mockapetris, P., "Domain Names Concepts and Facilities", STD 13,    
  568        RFC 1034, USC/Information Sciences Institute, November 1987.        
  570    [2] Mockapetris, P., "Domain Names Implementation and Specification",   
  571        STD 13, RFC 1035, USC/Information Sciences Institute, November      
  572        1987.                                                               
  574    [3] Partridge, C., "Mail Routing and the Domain System", STD 14, RFC    
  575        974, CSNET CIC BBN, January 1986.                                   
  577    [4] Gavron, E., "A Security Problem and Proposed Correction With        
  578        Widely Deployed DNS Software", RFC 1535, ACES Research Inc.,        
  579        October 1993.                                                       
  581    [5] Beertema, P., "Common DNS Data File Configuration Errors", RFC      
  582        1537, CWI, October 1993.                                            
  584 Security Considerations                                                    
  586    Security issues are not discussed in this memo.                         
  602 Kumar, Postel, Neuman, Danzig & Miller                         [Page 11]   

  603 RFC 1536            Common DNS Implementation Errors        October 1993   
  606 Authors' Addresses                                                         
  608    Anant Kumar                                                             
  609    USC Information Sciences Institute                                      
  610    4676 Admiralty Way                                                      
  611    Marina Del Rey CA 90292-6695                                            
  613    Phone:(310) 822-1511                                                    
  614    FAX:  (310) 823-6741                                                    
  615    EMail: anant@isi.edu                                                    
  618    Jon Postel                                                              
  619    USC Information Sciences Institute                                      
  620    4676 Admiralty Way                                                      
  621    Marina Del Rey CA 90292-6695                                            
  623    Phone:(310) 822-1511                                                    
  624    FAX:  (310) 823-6714                                                    
  625    EMail: postel@isi.edu                                                   
  628    Cliff Neuman                                                            
  629    USC Information Sciences Institute                                      
  630    4676 Admiralty Way                                                      
  631    Marina Del Rey CA 90292-6695                                            
  633    Phone:(310) 822-1511                                                    
  634    FAX:  (310) 823-6714                                                    
  635    EMail: bcn@isi.edu                                                      
  638    Peter Danzig                                                            
  639    Computer Science Department                                             
  640    University of Southern California                                       
  641    University Park                                                         
  643    EMail: danzig@caldera.usc.edu                                           
  646    Steve Miller                                                            
  647    Computer Science Department                                             
  648    University of Southern California                                       
  649    University Park                                                         
  650    Los Angeles CA 90089                                                    
  652    EMail: smiller@caldera.usc.edu                                          
  657 Kumar, Postel, Neuman, Danzig & Miller                         [Page 12]   

RFC9210 requires all DNS implementations to support TCP. In specific, it:

updates [RFC1536] to remove the misconception that TCP is only useful
for zone transfers.